I am new to Clover (using 2.9.4), and so ...
I have a UniversalDataReader taking in a CSV file in UTF-16 with | for delimiters.
The output ports are mapped to a DB derived metadata record with string, long, integer, boolean, and date fields.
I was having difficulty getting the parse to work, so rolled it all back to all string type fields and am moving forward with
each data type, one at a time. The boolean surprised me - only because the error message would indicate that my
specific case was omitted from the wide selection of options:
...(boolean) cannot be set to "False" - - doesn't match defined True/False format "T|TRUE|YES|Y|t|true|1|yes|y" / "F|FALSE|NO|N|f|false|0|no|n" in field # 5 of record # 2, value: 'False'
I have not found a way to allow this specific value to be valid, is there a way, or would my solution be to read this field as string and run it through a Reformat?
tia for any advice/clarification...
Rox
I have a UniversalDataReader taking in a CSV file in UTF-16 with | for delimiters.
The output ports are mapped to a DB derived metadata record with string, long, integer, boolean, and date fields.
I was having difficulty getting the parse to work, so rolled it all back to all string type fields and am moving forward with
each data type, one at a time. The boolean surprised me - only because the error message would indicate that my
specific case was omitted from the wide selection of options:
...(boolean) cannot be set to "False" - - doesn't match defined True/False format "T|TRUE|YES|Y|t|true|1|yes|y" / "F|FALSE|NO|N|f|false|0|no|n" in field # 5 of record # 2, value: 'False'
I have not found a way to allow this specific value to be valid, is there a way, or would my solution be to read this field as string and run it through a Reformat?
tia for any advice/clarification...
Rox
-
You may also parse the boolean value as a string value first (in Reader) and then in following Reformat component convert it to boolean using simple construct like this (in CTL1): $out_boolean:= iif($in_string == "True", true, false);
Which means that "True" value become boolean true and everything else will be considered false. -
thank you for this example of how best to do the conversion.
Strictly a theoretical question: do you know what the differences would be between iif() vs just == ?
( on the assumption that that inbound data stream really does give exactly "True", "False", or NULL for results)
e.g.
$out_boolean := iif($in_string=="True",true,false);
vs
$out_boolean := $in_string=="True";
If nobody's looked specifically - I wouldn't waste time finding out - but if there are error condition, validity, or performance issues that make a difference - that is worth knowing... thanks.
crrb -
Actually both cases are equal in terms of syntax/semantic validity. The " == " (plain equal) may be a bit faster. You may also use REGEX expression to capture variations of the "true" string. $out_boolean := $in_string ~= "(?i)^True.*";
Which captures any string starting with "True" (lower, upper or mixture cases) and anything after that. -
This is a short follow-up for those that haven't or are not going to v3 any time soon (since Agata indicates that this can be fixed via an option in v3). For version 2.9 at least I discovered quite by accident because of other research that the reader conditions for recognizing boolean values is stored in the defaultProperties file. So if you have a consistent set of values that you are looking for you can avoid the extra step in your graphs by customizing the defaultProperties file. Look for the Changing Default CloverETL Settings under advanced topics.
Please sign in to leave a comment.
Comments 5