Customer Portal

Global static variables

Comments 12

  • Avatar
    admin
    0
    Comment actions Permalink
    Hi yabel,

    I would suggest usage of dictionary - see http://www.cloveretl.com/documentation/ ... onary.html

    If you need both read and write it, it make sense to divide reading a writing components into 2 phases. Otherwise behavior won't be deterministic.

    Your CloverETL Support team
    Jarek
  • Avatar
    jus
    0
    Comment actions Permalink
    Hi Jarek,

    How can I use dictionary in the FileURL field of XLSDataWriter? I've read the docs here and here, but couldn't seem to get it to work.
    Also, I tried to use the dictionary in the command field of SystemExecute, and it didn't work. Please help.

    Thanks!
    Jus
  • Avatar
    jus
    0
    Comment actions Permalink
    Hi Jarek,

    How do I use dictionary entries in the FileURL field of XLSDataWriter? I tried the examples from here and here but it didn't work.
    Also, how can I use the dictionary in a commnd field of SystemExecute?

    Thanks!
    Jus
  • Avatar
    admin
    0
    Comment actions Permalink
    Hi jus,

    SystemExecute: You can prepare batch script executed by SystemExecute. Then pass arguments through input port to it - so for bash script it will be stdin. Then you need to parse stdin and launch commands you need. Here are some hints: viewtopic.php?f=4&t=5241&p=8231&sid=78de9d6fa4b335ca1a17520c79df67e1#p8231

    XLSDataWriter: The only way I know to pass dynamic file name is via "-P:name=value" to graph. There you can use ${name} substitution.
  • Avatar
    jus
    0
    Comment actions Permalink
    Hi Jaroslav, thanks for your help.
    I can continue working with the SystemExecute example.

    Though I'm not sure what to do with the XLSDataWriter, since the filename that I am writing to will be dependent on the output from the ComplexDataReader I'm using. Is there any way at all to do this besides passing a parameter to the graph? Because I don't have the value of the parameter before running it. Also, I can't use a higher level graph that calls RunGraphs because of the bug that was fixed in 3.3.0-M1, because we need this to be in production by the end of the month.

    I was thinking of writing to a generic filename, and then run SystemExecute to rename it. But can you think of another way?

    Thank you so much!
    Jus
  • Avatar
    admin
    0
    Comment actions Permalink
    Hi jus,

    generic file name is feasible if there is certainty that graph will not run in more parallel instances. So I would recommend to pass unique file name as graph parameter. Unfortunately there is currently no way how you can generate it by "randomUUID()" and use in both Writer and SystemExec on graph side.

    I can't think out better solution. In near, but not yet estimated future, there will be workflows support, which will solve issues you have.
  • Avatar
    jus
    0
    Comment actions Permalink
    Hi Jaroslav, thanks for your answer.
    My next question is, how should we handle those workflow things meanwhile?
    Should we use external scripts? Do it in the server? How do people do it usually? and what's your suggestion?
    Thanks!
    Jus
  • Avatar
    admin
    0
    Comment actions Permalink
    Hi jus,

    usually solution is based on:
    * SystemExecute (scripting extra logic)
    * RunGraph (run other graph somehow parametrized, possibly in loop)
    * HttpConnector (can invoke tasks on Server via simple HTTP API)

    All of them have currently some limitation, so it is necessary to use workarounds. That is the why we are preparing workflows support.
  • Avatar
    jus
    0
    Comment actions Permalink
    Thanks Jaroslav for your prompt reply! I really appreciate it.

    I now have a question about closing file handles.
    I am using ComplexDataReader to read in a flat file. After processing is finished, I'm using SystemExecute on the next phase to move the file into an archive or reject folder. But I get this error message:
    ERROR [SYS_EXECUTE1_0] - The process cannot access the file because it is being used by another process.

    I get the same problem when using XLSDataWriter and then trying to rename it using SystemExecute on the next phase (in the same graph).

    How can I close those file handles from within the same graph? Or is there any other solution you can think of?

    Thanks a bunch!
    Jus
  • Avatar
    admin
    0
    Comment actions Permalink
    Hi jus,

    ComplexDataReader closes opened file when graph is finished. That is not correct behavior and it will be fixed by: https://bug.javlin.eu/browse/CL-2229

    For now, only workaround is to use 2 graphs - first, which will run second and then move/delete file. In second will be complex data reader.
  • Avatar
    jus
    0
    Comment actions Permalink
    I guess it's the same problem with the XlsDataWriter? Will it also be fixed in 3.3.0-M2?
    Thanks again Jaroslav!
    Jus
  • Avatar
    admin
    0
    Comment actions Permalink
    Hi jus,

    sorry I missed that. There is new issue https://bug.javlin.eu/browse/CL-2235

    I would not expect it will be solved in 3.3.0-M2. The same may happen to CL-2229, although it is currently planned for M2.

Please sign in to leave a comment.