JSON reader mapping for Twitter JSON

sandeepbharmoria

• April 17, 2014 00:00

• Answered

Follow

Hi,

I am trying to fetch tweets and user details from tweeter JSON file.

For a simple JSON
{
"md5": "fa4c6baa0812e5b5c80ed8885e55a8a6",
"original": "example_text"
}

Mapping is :-

<context xpath="/root/object" OutPort="0">

<Mapping cloverField="md5" xpath="md5"/>
<Mapping cloverField="original" xpath="original"/>

</Context>

But what mapping should we do when there are other internal objects/arrays like

{
"metadata": {
"resulttype": "recent",
"isolanguagecode": "en"
},
"md5": "fa4c6baa0812e5b5c80ed8885e55a8a6",
"original": "example_text"
}

Please guide me on this ..

Thanks

Sandeep Thakur

• 

  imriskal

  • April 17, 2014 13:28

  0

  Comment actions Permalink

  Hi Sandeep,
  
  The nested information should be of course sent to another output port, it can be done like this:

  <Context xpath="/root/object" outPort="0">
    <Mapping cloverField="md5" xpath="md5"/>
    <Mapping cloverField="original" xpath="original"/>
      <Context xpath="//metadata" outPort="1">
        <Mapping cloverField="resulttype" xpath="resulttype" />
        <Mapping cloverField="isolanguagecode" xpath="isolanguagecode"/>
      </Context>
  </Context>

  Regards,
• 

  sandeepbharmoria

  • April 17, 2014 14:03

  0

  Comment actions Permalink

  Thanks a lot!!
  
  This resolved my problem..
  
  There is new one I encountered:- if there are special characters involved in the "values" like
  
  "source": "<a href="http://www.twitter.com" rel="nofollow">Twitter for Windows Phone</a>"
  
  ETL throws an error during reading the json file.
  
  Is there any way to escape these character "<" ">" across the file?
  
  
  Regards
  
  Sandeep Thakur
• 

  sandeepbharmoria

  • April 17, 2014 18:31

  0

  Comment actions Permalink

  Further please find attached, mapping.xml,test.json and error_logs for above mentioned error..
  
  Please guide me how to remove this parsing error..
  
  Thanks
  
  Sandeep

  • ctl_logs.zip
• 

  imriskal

  • April 18, 2014 08:40

  0

  Comment actions Permalink

  I am sorry but if the JSON file contains the line you posted, it is not a valid JSON file and our readers can not process it. The issue is not in angle brackets, it is in quote marks delimiting both strings and attributes of html elements. If Twitter returns such output, you could report the issue there.
  
  FYI, here is an useful tool for JSON validation: http://jsonformatter.curiousconcept.com/
  
  Regards,
• 

  sandeepbharmoria

  • June 10, 2014 09:21

  0

  Comment actions Permalink

  Hi,
  
  I have resolved this problem with following code...
  
  

  
  Context xpath="/root/array" outPort="0">
           <Mapping cloverField="id" xpath="id" />
           <Mapping cloverField="text" xpath="text" />
           <Mapping cloverField="created_at" xpath="created_x005fat" />
           <Mapping cloverField="number1" xpath="geo/coordinates[1]"/>
           <Mapping cloverField="number2" xpath="geo/coordinates[2]"/>        
  </Context>       
  

Please sign in to leave a comment.