Hello,
Is there any way we can limit the number of records which a graph can process?
Let's suppose we have a database table which contains about 5 million customers of the company and the graph database connector loads the data with:
But we want to "test" the execution of the graph to see if it fits the company needs, to do our tests we don't need to process all the 5 million customer, but just a part of it. I know that, in this sample, we could change the SQL query to limit the number of records loaded from the database, but this is just one sample, we have a lot of other data sources that aren't databases. Can we limit this record processing in some of engine initializer parameters or something like that?
Thanks in advance.
Luis.
Is there any way we can limit the number of records which a graph can process?
Let's suppose we have a database table which contains about 5 million customers of the company and the graph database connector loads the data with:
select * from customer where active = true;But we want to "test" the execution of the graph to see if it fits the company needs, to do our tests we don't need to process all the 5 million customer, but just a part of it. I know that, in this sample, we could change the SQL query to limit the number of records loaded from the database, but this is just one sample, we have a lot of other data sources that aren't databases. Can we limit this record processing in some of engine initializer parameters or something like that?
Thanks in advance.
Luis.
-
Hello,
all Readers, except DBInputTable and ParallerReader, have numRecords attribute, that specifies how many records/rows should be read from the source file. So you can use this parameter for testing purposes. For the real run of graph you can remove this parameter or set it to -1 - this means "process all records". Some Readers have numSourceRecords attribute also; it specifies how many records/rows should be read from every source.
Please sign in to leave a comment.
Comments 1