Minimize temp space requirements
Overview
Data Management can use a lot of temp space, especially if projects are not configured optimally. You can reduce Data Management's temp space usage by following these guidelines:
Don't use Left-to-Right merge
Using the Left-to-Right merge type in the Merge tool (formerly the Concatenate tool) forces Data Management to save records to disk that are not destined for the leftmost input. This uses a lot of extra temp space and reduces performance. Choosing a merge type other than left-to-right will greatly reduce temp space usage. See About the Merge tool for details.
Avoid sorting
Sorting uses a lot of temp space, and you should avoid it when possible. The following tools always produce sorted outputs. If you are connecting a tool downstream from one of these tools, and the sort keys are the same (or a subset), you may be able to eliminate a sort step,
Keep sorted data sorted
A common Data Management technique is to split data (using Filter or a similar tool), perform operations on the split data, and then merge it back together. If you choose a Left-to-Right or Greedy merge type, you may be losing an opportunity to preserve the record order and avoid downstream sorts. You can avoid this by configuring the Merge tool to bring the streams back together in order. For details, see Avoiding split-merge bottlenecks.
Configure Join tools correctly
If you are joining large record sets, and one of them is unique on the join keys but the other is not, connect the non-unique input to L and the unique input to R. This will run faster and reduce the temp space required by the Join tool.