Friday, October 17, 2008

Getting Some XML Love with libXML2

A while back Marcus Zarra did a good article on working with libXML2 and xmlTextReader to parse XML Data without loading the whole thing into memory. However the tutorial failed my needs only for 1 reason... he used a file on Disk. In his call to xmlReaderForMemory() he passes a path to the resource on disk (yes I know it makes for an easy self contained project that's easy to demonstrate, but bear with me here), I made the assumption that this made the method unusable for objects residing only in memory. In fact just about every method in libxml2 seems to have an argument for a path to a resource on disk and initially thought I couldn't really do strict in memory xml parsing. How wrong I was as you'll see later on. What I am going to show you today is how to create a valid xmlTextReader object from libXML2 with only the assumption that you are downloading the xml data from a server somewhere. Marcus showed a great intro into XML parsing with libXML2 and using a file on disk, but in my code I am getting Data back in memory from calls to web services on the internet and so I went to find out what it'd take to do XML Parsing with libXML2 without using a reference to a file on disk. Basically I wanted to assume that I just have a valid NSData object that has XML in it. When I got done with this I found out there are 2 ways to do this 1 very insanely easy way which isn't obvious from looking at the libXML2 method names alone and 1 harder way that accomplishes the same thing. The hard way

    1 NSData *xmlData; /* for this example assume xmlData has some valid xml data */
    3 xmlParserInputBufferPtr inpt = xmlAllocParserInputBuffer(XML_CHAR_ENCODING_UTF8);
    5 if(!inpt) {
    6     NSLog(@"Failed to create XML Input Buffer");
    7     return;
    8 }
   10 NSString *XML_STRING = [[NSString alloc] initWithData:xmlData encoding:NSUTF8StringEncoding];
   12 xmlBufferPtr xmlBuffr = xmlBufferCreateStatic((void *)[XML_STRING UTF8String] ,strlen([XML_STRING UTF8String]));
   14 inpt->buffer = xmlBuffr;
   16 xmlTextReaderPtr reader = xmlNewTextReader(inpt, NULL);
   18 if (!reader) {
   19     NSLog(@"Failed to create xmlTextReader");
   20     return;
   21 }
The point in this example is to create a xmlParserInputBufferPrt object that will be passed into xmlNewTextReader() so we don't read anything off of the disk. On line 3 we create this xmlParserInputBufferPtr by using the alloc method and passing in that we are going to use UTF8 encoding. And of course on line 5 we check for the existence of the xmlParserInputBufferPrt and if it doesn't exist there is really no point in going any further. Then I create a NSString object from the NSData object (line 10) which will allow us to get the UTF8String from the NSData object and (again) signify that we are using UTF8 Encoding. Then the big thing we need to create (line 12) is the xmlBufferPrt object. This creates a static buffer with the entire contents of the XML from the NSData object passing in a pointer to the string contents itself and the length of the string. Then (line14) in the xmlParserInputBufferPtr we point the data buffer pointer to the xmlBufferPrt object we created on line 12. After that it's just a matter of creating a xmlTextReaderPtr (line 16) with xmlNewTextReader and pointing the input buffer to the xmlParserInputBufferPointer which now has all the XML in it and pass NULL to the path of the XML. Now you have a xmlTextReader which you can use to parse xml with. The Easy Way Now I started going down this path because of Peter Hosey's suggestion of using xmlParserInputBufferPtr which logically seemed like the best solution to my problem of wanting to use strictly in memory objects and do no reading off of disk. If I could just pass NULL for the path in xmlNewTextReader() could I do the same with xmlTextReaderForMemory and it'll still work? As it turns out... YES... yes it does work.
    1 NSData *xmlData; /* for our purposes here assume xmlData has valid xmlData in it */
    3 xmlTextReaderPtr reader = xmlReaderForMemory([xmlData bytes], 
    4                                                                                          [xmlData length], 
    5                                                                                          NULL, NULL, 
    8 if (!reader) {
    9     NSLog(@"Failed to create xmlTextReader");
   10     return;
   11 }
In fact all you need to do is set the 3rd and 4th parameters to NULL in xmlReaderForMemory() and it works fine with in memory objects like if you have a NSData object you got back from API's like say NSURLConnection sendSynchronousRequest. The only thing I wish is that there was a lot better documentation on libxml2, maybe there is a great rescource I just don't know about, but from my googling it was hard to find anything and I had to do a lot of trial by error. Update: Im aware of the documentation at, most of it however pretty much just shows you a list of method names and describes the arguments you pass in to methods with a couple decent doc's/tutorials. This isn't great documentation for me as it doesn't describe which arguments are necessary, if a argument is optional you should make that crystal clear and the documentation doesn't make it clear that a lot of arguments are in fact optional in libxml2 methods. Specifically one page im referring to is at

Wednesday, October 01, 2008

Thank Goodness the F'ing iPhone NDA is being lifted

Apple FINALLY did the right thing today and publicly recognized what pretty much all iPhone developers and the public that have been paying attention to the news have known for a long time now, that the iPhone NDA was doing much more harm than good. From their page ( ) "To Our Developers We have decided to drop the non-disclosure agreement (NDA) for released iPhone software. We put the NDA in place because the iPhone OS includes many Apple inventions and innovations that we would like to protect, so that others don’t steal our work. It has happened before. While we have filed for hundreds of patents on iPhone technology, the NDA added yet another level of protection. We put it in place as one more way to help protect the iPhone from being ripped off by others. However, the NDA has created too much of a burden on developers, authors and others interested in helping further the iPhone’s success, so we are dropping it for released software. Developers will receive a new agreement without an NDA covering released software within a week or so. Please note that unreleased software and features will remain under NDA until they are released. Thanks to everyone who provided us constructive feedback on this matter." Personally I am of the opinion that it's far better for people to eventually recognize that they made mistakes and try to correct it than people to live in blissful ignorance and just believe they did the right thing, so in this regard I am glad that Apple has finally publicly come around and recognize that the iPhone NDA, while it helped Apple "protect" some things on the iPhone, it was really doing a net damage to the platform that hurt it and made developers afraid to even touch the iPhone SDK. I was really looking forward to Bill Dudney's Core Animation book, I really originally just wanted to use it on the Mac, but because it just contained 1 chapter on the iPhone Core Animation differences the Pragmatic Programmers couldn't publish it. Thats 3-4 months that I couldn't simply hold a book in my hands and learn something that contributes back to Apples platform because 1 small part contains something on the iPhone (Yes I have the PDF, but I find it hard to sit down at a computer and read a whole book that way, I really only read book PDF's on my Mac as a reference looking up one small piece of information I want.) Now I hope they rescind what they said in their recent email saying they would just publish it without the chapter and finally publish the whole book in its entirety. And it's not just that the Core Animation book was probably one of the most publicly known books initially that people knew was being held up by the NDA, Amazon shows many other books on iPhone Development that are not yet available due to the NDA. In the end I really hope Apple has learned a lot from this. Now I hope that iPhone App quality will go up as a result of developers and sites soon being able to finally share code and information between each other. I know I have a couple iPhone SDK articles that are in development, but I haven't given them the time they deserve because I just didn't know if Apple was ever going to lift the NDA anytime soon. Apple we love you, we love the Mac and the iPhone, and we are even willing to put up with some things that developers of other platforms would scoff at because we like this platform that much. But when you get a ton of negative press and tons of developers are very publicly and loudly criticizing you all with the same opinion, it's not to give you grief, it's because we care about this that much and you're royally screwing up on something. If we really wanted to hurt you we'd be silent and say nothing. Also where is my iPhone Dev Key Apple?