I have a graph with 5 db executes in parallel being fed with the names of scripts to execute. Coming out of them, I have a reformat to capture any errors that happen and feed that into a log table. When I try to run this, I start getting Direct Buffer Memory errors. I set -Xmx384m and the error went away, but the graph failed stating it cannot find an object that one of my scripts creates. After increasing to -Xmx768m, the graph runs successfully.
I have 2 questions:
1. why does such a graph require so much memory? Is there anything I can do about that?
2. if it really is needing so much RAM, why didn't I get the Buffer Memory error when set at 384m? -- it was like it gave me false error message with that setting.
I have 2 questions:
1. why does such a graph require so much memory? Is there anything I can do about that?
2. if it really is needing so much RAM, why didn't I get the Buffer Memory error when set at 384m? -- it was like it gave me false error message with that setting.
-
I have included 3 traces, and 2 sql files. Each trace file is named according to how much memory I allocated for that run. The 2 sql files are the ones it usually complains about, but they should be un-related to each other (no dependency between them). -
Hello,
problem is probably in the very long transactions processed on the database.
I've found two interesting articles on the internet:
I believe, that changing the transactionIsolation in your connection, could help. -
I did some more work on this, and discovered I had some weird interaction going on with the sql scripts I was running that should not have been happening. They were both calling a procedure to create some indexes, and that called procedure was causing my locking issue.
That bit is taken care of, however, I still see some really high memory requirements for many other of my graphs. Is there any general help you can offer in tracking down memory usage issues? -
There is only few general advices how to decrease memory requirements. First of all try to not use external jars in jdbc, jms connections - rather add them to the classpath when running graph. Also some components are memory gluttons and often can be replaced by others, that don't require so much memory. Some components can be adjusted by proper attributes values (see eg. ExtSort vs. FastSort – which one is better for me?). But, when you suspect a graph to require too much memory, it requires an analysis for this concrete graph.
Please sign in to leave a comment.
Comments 5