In my job flow, I have multiple nested graphs. In one of the nested graphs I perform a rest call that generates an unique identifier. Ideally, I would want to store this unique id as a global variable of the current run so that I can easily access this in other graphs within the same job flow.
Dictionaries seem like a good solution except they only work within a single graph. Another option is an external lookup table however I only need to store a value, not a key.
Is this the right way to go about this? what do you guys suggest?
-
Your approach with external lookup table is possible.
However, a better approach is through dictionary entry and JobFlow graphs. You can use the pair of SetJobOuput & GetJobInput components and the fact that when you call a (sub)graph, then you can map-out dictionary entry which that subgraph changes and use it in the calling graph.
So you top level JobFlow would first call that graph which performs that REST API call and gets that one value. That value can be stored into dictionary entry, which, after that graph finishes, available to the calling graph. From there, you can pass that unique ID to other transformation graphs.
Please sign in to leave a comment.
Comments 1