Customer Portal

Clover graph and temp space

Comments 2

  • Avatar
    avackova
    0
    Comment actions Permalink
    Hello Ann,
    thees files are temporary files for sorting, joining and for phase edge's data and they should be removed after requested phase has finished. If the files exist after graph execution it is a bug, but we have never been able to reproduce the issue.
    In components you can change temporary directory by setting proper attribute, but it is unfortunately impossible for phase edges (http://bug.cloveretl.org/view.php?id=2541).
  • Avatar
    anweston
    0
    Comment actions Permalink
    Heya,

    Going back through the posts on this subject, it looks like I can divert most of the temporary files (the .fb*) by adding the following flag to the JVM upon startup:

    -Djava.io.tmpdir=/wherever_I_want

    I plan to do some more testing, but this should not do any harm, right?

    I've found that our IT group usually allocates about 6G to /var. My 5 graph test using a small-to-large dataset (for our data) caused about 3G worth of temp files, so it's not going to be enough when we run really big datasets.

    Thanks,
    Anna

Please sign in to leave a comment.