Stay up to date on the latest product updates, customer stories, and industry news from Redpoint. Opt-in and select the topics you'd like to learn more about!
Redpoint Data Management™ Documentation
Redpoint Data Management™ Documentation
Breadcrumbs

Configure Machine settings

Overview

To view the settings for a computer, open the Machines folder under Settings in the repository. Select the desired computer name, and then go to the Properties pane.

All controls are disabled unless you explicitly override the Site Settings defaults.

  • To change a setting, select Override.

Tuning

The Tuning settings control the number of threads that projects use and general memory allocation. Use them to tune project performance for a particular computer configuration.

  • Tool threads: the number of threads available to tools during project execution. This is generally set to equal the number of CPU cores on the computer (x2 if hyperthreaded).

  • Memory limit (MB): defines the memory limit per project in megabytes.

  • Memory aggression: controls how quickly memory is allocated.

    • The default setting is 30.

    • Set a lower number if a project produces warnings about tools getting less memory than requested.

    • Set a higher number to optimize the performance of Sort and Join tools within a small project.

  • Sort parallelism: setting to two enables parallel sorting operations, which can improve the performance of certain tools and macros under some conditions. Set this equal to the number of defined temp spaces, but no more than two. See Sort parallelism for details.

  • Enable hidden Data Viewers: tools that are embedded in projects or macros are disabled by default. Select Enable hidden Data Viewers to view these tools.

CPU cores and hyperthreading

When using Redpoint Data Management (RPDM) on virtual machines (e.g., in Azure, AWS, or VMware), the RPDM Management Dashboard may report only half of the total CPU cores available on the machine. For example:

VM Total vCPUs

RPDM Cores Detected

2 vCPUs

1 core

4 vCPUs

2 cores

8 vCPUs

4 cores

This behavior has been observed on RPDM for Linux running in Azure environments, but applies to all platforms where the underlying CPU architecture supports hyper-threading.

This is expected behavior and is by design. RPDM counts CPU cores as follows:

  1. RPDM first counts the virtual CPUs (vCPUs) reported by the operating system

  2. If the CPU architecture is hyper-threaded: RPDM cores = virtual CPUs / 2

  3. If the CPU architecture is NOT hyper-threaded: RPDM cores = virtual CPUs

Most modern processors (Intel Xeon, Intel Core, etc.) and most cloud VM types in Azure, AWS, and VMware use hyper-threaded architectures. In these cases, each physical core presents two logical threads (vCPUs) to the OS. RPDM divides by 2 to reflect the actual physical core count, since performance does not scale linearly with hyper-threaded logical cores.

What Is a "Virtual CPU"?

Environment

vCPU Definition

Virtualized (VMware, AWS, Azure)

Determined by the VM definition/SKU

Bare-metal with hyper-threading

Number of CPU threads

Bare-metal without hyper-threading

Number of CPU cores

Diagnostics

These are generally specified on the Site Server, but Redpoint Global Inc. Support may direct you to enable these on a per-computer basis. See Site settings - Diagnostics.

Output files

“Open Output files early” is the sole option in this section.

Selecting “Open Output files” early causes file output tools to open their output files immediately when the project starts, rather than waiting for the first record to be written to to the file.

  • Select this option if you have long-running projects and don't want to wait for the first record to reach a file output tool to validate that the file is writable.

  • Do not check this option if you are reading and writing the same file in the same project (reading a file, modifying it, and re-writing the same file).

The Xml Output2 tool always opens its output file when the project starts.

Testing

The Testing section contains options controlling a test mode of project execution. This is primarily intended for project-level configuration and should rarely be changed at the server level.

Messages and Database

These are generally specified on the Site Server. See Configuring Site Settings.

Machine license

The “Limit cores” option lets you control the number of CPU cores allocated from the license for a specific Execution Server. Execution Servers normally allocate the physical CPU cores on a first-come/first-serve basis. If the servers collectively have more CPU cores than are available in your license, one or more of your Execution Servers will be starved of some or all CPU cores.

You can limit the number of CPU cores that each server gets by selecting Limit cores and specifying the maximum number of Cores to license for each server.

For example, if you are licensed for 16 CPU cores, and you have two 16-core Execution Servers, normally the first one connecting to the Site will get all of the cores and the other will get none. However, if you select Limit cores and specify Cores as 8, Data Management will distribute the licensed CPU cores evenly between the servers.

Resources tab

See Configuring temporary disk space for information on configuring temp space.