duplicate symbol errors with third party library

We use different commercial third party libraries when making iPhone applications. One of them got an update a few weeks back, but when we tried to integrate it into our application xcode threw out the following errors (actual library name hidden);

duplicate symbol _FBLoginViewButtonPressedPNG_retina in:
    SSSSS.embeddedframework/SSSSS.framework/DDDDD(FBLoginViewButtonPressedPNG.o)
    ./FacebookSDK.framework/FacebookSDK(FBLoginViewButtonPressedPNG.o)
duplicate symbol _FBLoginViewButtonPressedPNG_standard in:
    SSSSS.embeddedframework/SSSSS.framework/DDDDD(FBLoginViewButtonPressedPNG.o)
    ./FacebookSDK.framework/FacebookSDK(FBLoginViewButtonPressedPNG.o)
duplicate symbol _OBJC_METACLASS_$_FBLoginViewButtonPressedPNG in:
    SSSSS.embeddedframework/SSSSS.framework/DDDDD(FBLoginViewButtonPressedPNG.o)
    ./FacebookSDK.framework/FacebookSDK(FBLoginViewButtonPressedPNG.o)

At first glance, the problem appears obvious. We are (inadvertently) including Facebook’s libraries twice and the compiler (or rather the linker) is confused which reference to use. The difficulty is that we only included the Facebook library once. So where was this second reference coming from? Did we accidentally include it without realizing it? A very far-fetched theory, but it wouldn’t hurt to check, so we removed the one reference we knew we included and tried building the application again.

Not surprisingly, we got a lot of “missing ABC” errors this time.

A bit of thought led to the conclusion that our third-party library might be embedding the facebook libraries within itself. The theory made sense – the problem started happening after we tried to integrate an update to their library.

Deciding that the problem was isolated, we reached out to our third-party library vendor for assistance. Initially, we got responses from them that the issue was under investigation. Then a week passed. Two weeks. I even got engaged. And the problem still persisted.

Quite frankly, I was more than willing to throw away the so-called “update” to the library and just use the older version. That at least, worked with our code. Unfortunately, the customer we were working for insisted to use the newer version. The old version you see, was meant for iOS6. With all the visual differences between iOS6 and iOS7, the interface didn’t really mesh well.

So what was left to do? The issue was caused by the third-party vendor’s new update, but they wouldn’t give us a timely fix. What options were left open to us? Throw our hands up in despair?

I was literally, at the end of my rope. Stuck between a rock and a hard place, I asked myself – would it be possible to manually remove the references to facebook’s API from the third party vendor’s library? Reverse-engineering the binary would probably by a violation of the Terms Of Service we agreed on when obtaining the library, but I was really desperate here.

So I tried it.

First step was to find out what exactly I was working with. For this, I used the “lipo” tool to analyze the binary library our third party vendor provided.

$ lipo -info SSSSS
Architectures in the fat file: SSSSS are: armv7 armv7s i386

Hmm… the library comes with 3 different architectures bundled in? Better to check them out one-by-one.

$ lipo -thin armv7 SSSS -output SSSS-armv7

What the above command does, is to split the static library named “SSSS” into a separate file for the architecture “armv7” (works because SSSSS is a fat file).

Next we need to analyze the file to list out its included object files. The “ar” tool comes to our rescue here.

$ ar -t SSSSS-armv7
__.SYMDEF
A2BlockInvocation.o
A2DynamicDelegate.o
AFHTTPRequestOperation.o
AFHTTPRequestOperationManager.o
AFHTTPSessionManager.o
AFNetworkActivityIndicatorManager.o
AFNetworkReachabilityManager.o
...

A little down the list I came across these tell-tale entries,

...
Facebook.o
FBGraphObjectTableDataSource.o
FBRequestBody.o
FBSystemAccountStoreAdapter.o
FBFriendPickerViewController.o
FBAppEvents.o
FBFetchedAppSettings.o
FBUtility.o
...

Looks like the Facebook API to me!

Having identified the source of all the “duplicate symbol” problems, the next step is to get rid of them. For this, I created a new empty directory and unpacked the object files from the archive into it.

$ mkdir SSSSS-armv7.folder
$ cd SSSSS-armv7.folder
$ ar -x ../SSSSS-armv7

All that’s left to is to delete the problematic object files from this folder and repack the archive, like so;

$ rm FB*.o
$ libtool -static *.o -o ../SSSSS-armv7

You can use the “ar” tool to reconfirm that the libraries have been removed.

At this point, only the armv7 architecture had been cleaned. The above process needed to be repeated for all the architectures present in the third-party vendor’s library.

The second-to-last step is to recombine all the thin files into a big fat file again (for xcode to use).

$ lipo -create SSSSS-armv7 SSSSS-armv7s SSSSS-i386 -output SSSSS-noFacebook

Final step was of course, to attempt using this “reduced” library with our application in xcode. Happily enough, it worked! As a developer I was so relieved that it all came together – after so much pain too. And obviously, our customer is happy as well. As for the third-party library vendor? Who knows whats going on with them? I’m still a little annoyed that they put me through all those weeks of pain.

References:

Avoiding duplicate symbol errors during linking by removing classes from static libraries
CocoaPods Troubleshooting

Windows 8.1 upgrade – odd behavior

For the past week (possibly more), my laptop has been taking a very long time to shut down. By itself, that is an odd thing to be complaining about, but I’m the kind of person who likes looking at details and in this specific case, just have a tendency to push the “off” switch on the power socket when I’m done with my system. Wanting to save a bit of time I decided to search up on the matter. Here are a few links that I found;

Clearly, I’m not the only one facing this issue. But complaining about windows happens so often, that the result is not that surprising. No, what I found surprising (the point of this post!) and perplexing was one of the suggestions I came across – to make a new user account.

I had to pause for a few moments when I saw that. It had to be a joke, right? But reading further on the thread (not sure if its among the lot above), the poster was serious. He(/She) claimed that it was due to some corruption that happens to a user’s account when upgrading and make a new user from scratch solves the matter. Between the other suggestions of rolling back drives, or even the OS update itself, this was a relatively cheap suggestion and I gave it a shot.

I opened up the control panel, made a new “temp” user and gave him administrative privileges. Logged out of my current user-profile, logged into the new one and tried turning off my system. It still took a long time to switch off. Huh. So much for that theory. Still, I wanted to try another experiment. I rebooted into my laptop and directly logged into the “temp” user without touching my original account. Turning off my laptop this time was a lot faster. In fact, it is comparable to how things were for me pre-win8.1 update.

What is causing this behavior? I wish I knew how Microsoft handled user profiles, but since I don’t work there, it’s not in my hands. What IS in my hands, is the rather bothersome task of moving my files to the new user account – and deleting the old one afterwards. At the time of typing, I’m actually finished with the move, but am going to keep the other account around for a bit longer, in case I forgot something. Will probably delete it from my system after a month (Why leaveĀ  an unused account on the system, anyway?)

The new account also revealed a few odd quirks with Windows8.1. The text in Google’s chrome browser for instance became rather blurred. Let me try pasting a screenshot;

BlurredChrome

On seeing this, I began to wonder if this was Microsoft’s latest attempt at getting people to use Internet Explorer again. Conspiracy theory or not, I needed to fix this and went to the internet (blurry browser in tow) to find answers. It took me awhile, but I finally narrowed down the problem to a Display Scaling setting. Let me describe the answer first and then come round to proposing an explanation. You need to find the location where the chrome executable is stored. Open up Explorer and navigate to C: > Program Files (x86) > Google > Chrome > Application > chrome.exe. Right-click the executable and select properties. Go over to the ‘compatibility’ tab and check the “Disable display scaling on high DPI settings” option (like figure below).

compatibility_options

On restarting chrome this is what I got,

BetterChrome

I’m not sure if you can see the difference with the screenshots, but to my eyes it’s a whole lot better. Oddest thing, is that this isn’t a chrome-only problem. I’m facing the same issue (and solution) with a couple of other applications too – Keepass being the most recent example. Weird.

Asynchronous programming – breaking the illusions of blocking code

Over the past couple of weeks, I’ve been forced to confront the asynchronous programming style. Not out of choice, but because it was the only interface provided by a third-party library that needed to be integrated. Now before I begin, let me stress that asynchronous does not mean parallel programming. If this were parallel programming, I could try wrapping the offending code in some special context and most problems would go away. No, asynchronous is actually single threaded execution pretending to be parallel.

Let’s start off with a simple code snippet to understand this. Have a look at the pseudo-code below;

for (i = 0; i < 3; i++ {
    print("current loop=" i);
}

The output of the above program would look like this;

current loop=0
current loop=1
current loop=2

For the most part, the flow of execution is straight-forward enough, isn’t it? Now, remember that 3rd-party library that I needed to work with? Well, that was an asynchronous call that returned a value that needed to be used inside the for-loop. Naively, the following attempt at integration was made;

for (i = 0; i < 3; i++ {
    async_print("current loop=" i);
}

The result baffled me. Here’s what I got;

current loop=3
current loop=3
current loop=3

Which didn’t make any goddamn sense! … at first. I spent a considerable amount of time searching for answers. In the end, it came. The problem was that I didn’t quite understand the flow of program execution.

Coming from a synchronous programming, I assumed that asynchronous calls were some fancy way of parallel programming. They aren’t. I also assumed that the async function call would be blocking (i.e. the program would wait till the function/method finished executing). That wasn’t quite the correct answer either.

You see, what happens with asynchronous statements (at least the ones I’ve encountered), are something what might be called as “deferred execution”. When an asynchronous statement is reached, the control flow does not execute it. Instead, the statement is put on a queue to be executed later. Once the current context of program statements are finished, items in the queue are taken out in a FIFO manner and executed.

So, how does this explain the earlier for-loop? Within each loop, when the async call is made, its put on the queue. Now, when will the loop be over with? When the value of i is equal to 3. It is in this kind of context that the async calls get serviced. So, even though they are called three times, the value of i for each of those calls is 3 – thus producing our strange result.

I’m probably missing out some important details, but from an understanding point of view, this know-how was critical to help me solve my problem. The solution I resorted to was recursion. Here’s how the ‘fixed’ pseudo-code looks like;

i = 0;
function (i)
{
   async_print("current loop=" i );
   i++;
   if (i < 3)
      call function (i);
}

Things might seem a bit stretched out, but the key concept that needs to be understood, is that now every call to function() happens in a different context. This means that even if the code flows by skipping the async_print() function initially, when it returns to service the call, the context to which it returns is the same. Thus giving us this output;

current loop=0
current loop=1
current loop=2

Hmm… this would probably be better if I could demonstrate some actual code instead of this pseudo-stuff. Let me see if I can rig up something …

Configuring Doxygen with github

I had heard that it was possible to setup projects in github and have the corresponding Doxygen output available online. A quick search of the terms brought me here,

http://rickfoosusa.blogspot.in/2011/10/howto-use-doxygen-with-github.html

The instructions seemed simple enough and I decided to make my own account on github. It took me a few hours but here’s my test repository,

https://github.com/abrahamvarricatt/test

And here is the corresponding documentation,

http://abrahamvarricatt.github.io/test/

There’s not much to see in the way of documentation and the source code itself is a basic “Hello World” application that I shamelessly copied from somewhere. Continue reading

container_of() gcc macro

A colleague of mine approached me a few days ago with the following question – “What does container_of() do?”. He linked to the following file in the Linux kernel (we do embedded development at work),

http://lxr.free-electrons.com/source/scripts/kconfig/list.h#L18

/**
 * container_of - cast a member of a structure out to the containing structure
 * @ptr:    the pointer to the member.
 * @type:   the type of the container struct this is embedded in.
 * @member: the name of the member within the struct.
 *
 */
#define container_of(ptr, type, member) ({          \
    const typeof(((type *)0)->member)*__mptr = (ptr);    \
             (type *)((char *)__mptr - offsetof(type, member)); })

At first glance, I wondered what kind of function was I looking at? It soon became clear that it wasn’t a function at all, but a macro. One of those compiler directives the kernel developers enjoy using.

Continue reading

Flashing a new kernel and ROM on an Xperia Arc

While there are a plenty of tutorials on how to flash a new kernel/ROM onto my phone available on the XDA developers forum, fact of the matter is that I don’t visit very often. The result is that every few months when I feel like ‘upgrading’ my phone, I have to read/search through a mountain of posts to get the information I need. Being the lazy kind of individual, I’ve decided to make my own notes on how to do the upgrade. For anyone reading this, please note: these instructions are for an unlocked Xperia Arc (international release).
Continue reading

disabling the android lock screen & welcome message

I build android systems from source often enough these days.; usually to test some new feature or the other. One of the first things one can do with a new android system is tap the ‘OK’ button on the “Welcome to android” tutorial that comes up. After doing this for the umpteenth time, I had enough! And that silly lock screen too! There had to be a way to statically disable these annoyances in source code, and I was going to find it!

Fortunately, it wasn’t as difficult as I thought it would be.

Continue reading