Wednesday, April 17, 2013
Difference between server jobs and parallel jobs?
Server jobs:-
1)In server jobs it handles less volume of data.
2)It is having less number of components.
3)Data processing will be slow.
4)Executed by Datastage serve engine
5)compiled into Basic
6)No parallel Capability is one of the drawback of Server jobs
Parallel jobs:-
1)It handles high volume of data.
2)Executed by Datastage parallel engine
3)It is having more number of components compared to server jobs.
4)Supports pipeline and partition parallelsim
5)Compiled into OSH
Similarity Between Server jobs and Parallel jobs:
Runtime monitoring in DataStage Director
Tuesday, April 16, 2013
COLUMN GENERATOR STAGE
1)The Column Generator Stage is a development/debug stage.
2)It can have a single input link and a single output link.
3)The Column Generator adds columns to incoming data and generates mock data for these columns for each data row processed.This is useful for testing a job when no real test data is available.
ROW GENERATOR STAGE
1)The Row Generator Stage is a development/debug stage. The stage has no input links and only one output link.
2)The row generator stage generates a set of mock data fitting the specified metadata. This is useful for testing a job when no real test data is available.
3)Metadata can be specified using a schema file.
Sort stage
1)Sort stage is a processing stage used to perform sorting operations on input data.
2)Need for Sorting:
Some stages require sorted input
ex- Join, merge stages
3)Sort stage requires a ‘key’ to be specified by which the sort is performed.Multiple sort keys can be specified
4)Sort operation is performed partition wise.To sort a complete set of data, you should change the Sort Stage execution mode to sequential.
5)There are two ways Sort can be performed in Datastage :
a)Within stages On input link Partitioning tab, set partitioning to anything other than Auto
b)In a separate Sort stage which has more options like Allow duplicates,case sensitive,sort order(ascending / descending) etc.
By default, both methods use the tsort operator which can be identified in Job score
6)Partitioning keys are often different than Sorting keys
Keyed partitioning (e.g.Hash) is used to group related records into the same partition where as Sort keys are used to establish order within each partition
2)Need for Sorting:
Some stages require sorted input
ex- Join, merge stages
3)Sort stage requires a ‘key’ to be specified by which the sort is performed.Multiple sort keys can be specified
4)Sort operation is performed partition wise.To sort a complete set of data, you should change the Sort Stage execution mode to sequential.
5)There are two ways Sort can be performed in Datastage :
a)Within stages On input link Partitioning tab, set partitioning to anything other than Auto
b)In a separate Sort stage which has more options like Allow duplicates,case sensitive,sort order(ascending / descending) etc.
By default, both methods use the tsort operator which can be identified in Job score
6)Partitioning keys are often different than Sorting keys
Keyed partitioning (e.g.Hash) is used to group related records into the same partition where as Sort keys are used to establish order within each partition
Subscribe to:
Posts (Atom)