Patch Coloring: A way around the proxy

There has to be a name apart from “procrastination” for the phenomena that makes you delaying writing up a post. Time multiplies exponentially under the thrust of accomplishing more so that can also be included in the write up.

Anyways, in the last post we left our project at an interesting point where we got to know that File Descriptors of the VPN may not let us capture the entire traffic on device and we did not want to do it the proxy server way. So, what next?

Good news is that we were able to capture the device traffic using VPN.

What we did is essentially fitting the VPN to capture the device traffic rather than adding a proxy manually or programmatically using root.

My first attempt was to connect VPN to a nanohttpd server, but then I found that was not needed.

Make a tunnel to connect to any random port and configure the VPN to get the ParcelFileDescriptor.

 Builder builder = new Builder();
 builder = new Builder();
 builder.setMtu(1500);
 builder.addAddress("10.0.0.2", 24);
 builder.addRoute("0.0.0.0", 0);
 
 mInterface = builder.establish();

This helps us capturing the traffic and we got packets like this one:

malformedPacket.PNG

Now what? We needed to make sense of this packet, so we extracted crucial details from it and got something like this:

ipPacketExtractInfo.PNG

Cool! So we have the requested traffic but now how to fulfill the requests? Where should we direct them to and how should we connect to the internet to get them addressed?

We employed simple JAVA sockets to accomplish this task. If you rightly remember, we had a proxy server which was responsible for listening to incoming requests and fulfilling them using ProxyConnectionHandler and then writing back to output.

But what is the nature of these incoming requests? Here comes an interesting fact that has to do with the OSI layers.

The request we got at the proxy server end were like these:

proxyRequest.PNG

This is a typical HTTP request with following structure:

httpreq

While the one we receive from VPN interface file descriptor are purely the IP packets.

Now comes the interesting part, and the mistakes made.

Here is what I was trying to do:  What VPN essentially done is it creates a tunnel, that directs all network traffic of the device to that tunnel. The VPN interface is essentially a parcel file descriptor and one can use FileInputStream and FileOutputStream to write in /out the file descriptor.

The intention was to direct the packets received from the fd(file descriptor) straight away to the proxy server(we used before) by using JAVA sockets. And from that proxy server the rest of the logic will follow. But the things we now observe were rather interesting.

  1. The requests entered into a loop. Here’s how: You trace the requests by setting up VPN on the device, now you want to fulfill these requests so you pass them to proxy server that in turn tries to make connection to the URL host, but now since the entire device is under VPN enabled network, these requests also get passed to the VPN interface and the loop continues, in short there was no way to reach to the internet out there in that scenario.

Solution: Protect the socket from VPN! Yes, we can do that using VpnService.protect(socket). So now only this socket gets out of the VPN control and can connect to the underlying network.

2. We saw certain packets coming across this socket now(that is protected) but still no rendering, why?

Reason might be  the OSI layers we are operating into here. We captured the IP packets , however for transmission over sockets, we need TCP/UDP so we need to form the packet properly, may be add the required headers.

osi.PNG

Until that is done, the packet will remain malformed and could not be responded  well by the hosts. How that is to be done is still my quest though.

The third thing that might turn up is writing these requests back to the File Descriptor. So, essentially, these things were done internally by the device when we added a proxy server manually, making the packets ready for transmission over sockets, but presently we are one layer deeper and need to add the requisite to come to the layer sockets operate at.

Another interesting aspect I am pondering on presently is how to track from which app the traffic is coming. Let’s see how it turns out to be.

Helpful link: http://stackoverflow.com/questions/20237743/android-firewall-with-vpnservice

 

Too much for a post? May be 😛 Lemme add some music. This time we have FIFA 2016 tracks.

So, see you in the next post, will make it sooner, till then keep exploring 🙂

 

 

 

 

 

 

 

Choosing colors: Hits & Trials

This post was intended a week ago but then the week just happened and now here I am in the middle of the night jotting down the happenings with my Outreachy project.

In the last post (lemme check) we talked about basic app architecture and I tried explaining the logic of the app in the prior posts and what we intend to do.

During this tenure my time and energy has been devoted to contemplating on technical situations the app can face and avoid it from crashing, managing consistency with the database for it was shared by the application and background service for reads and writes, picking up proper data structures and scheduling the reads and writes in batches to save time, considering the maximum space a running app may need , how should Privacy Badger get the desktop browser filter settings, how it intends to update them periodically, and discussions on several other crucial points including the implementation of logic to identify trackers.

Rest was devoted on how to intercept the traffic without directly modifying the wifi setting to add proxy server (127.0.0.1:8090) manually. The suggestion was to understand how androidVpnService helps making that possible.

I studied apps like disconnect that uses openVPN and orbot that provides tor for android.

Till now its been figured out that while it is possible to direct one app’s traffic to a proxy server, getting a system-wide proxy on android has no simple way.

The strategy I tried was having a proxy server in place on the device i.e. locally hosted, the app will make a vpn connection to this server and the server will then connect to the internet. The bidirectional flow, http/https requests and logging the count per URL in the database is all been tracked at the proxy server’s end.

Also, it consumes resources of the system to maintain a consistent VPN connection too.

Since there is no way yet employed to create a system-wide proxy on the android device programmatically, presently the aim is to figure out other possibilities for the same , and if not make it app-specific. Once that is figured, then comes the tracker logic in picture.

Hoping that the issue is resolved soon and the next post hits the blog within few days describing the route we have finally chosen to move ahead with.

Here are couple of screenshots of the application presently, the right pane shows system wide URL requests been tracked:

 

So that has been all about work, might have missed out on finer details, I will try to cover them in the coming post.

Till then, keep hacking, stay secure , meanwhile I am like where is my cup of:

0044

 

MUSIIC: Eddie Vedder

Adieu

 

 

 

 

 

 

 

 

Done with sketching: App architecture

Hola folks!

Long time and a delayed update. It takes real motivation to track down your own progress than to actually code and develop the application.

code

So what have I been up to? This time period ended up with a ready architecture for the application with primarily three blocks:

1. UI : Where the user interacts, for now we have just two activities

(i) Here user can turn on/off  PB, the tracking service.

(ii) View data: Tracked URLS, frequency count and status. The view is generated by fetching content from db.

2. Service that runs the server: Android service makes sure that PB keeps on running in background though app is closed by the user, until user manually sets it to off. This section is devoted to keeping track of requested URLs and making necessary updates to database.

3. Database handler: This section provides handy functions to store and access URL data easily on the device and is responsible for overall data management.

As of now, section 1 and 3 are almost complete. The most important component is to run the service efficiently. For now I am linking it to results fetched by proxy server only, to check it is working fine, but it has to be substituted with androidVpnService next.

Since, running proxy needs root access and we intend to avoid that.
So, done with the basic architecture of the application. Major focus now is to improve it and make sure the app never hangs due to increasing frequency of tracked URLS.
Once, done with that performing the tracker service using  androidVPNService instead of currently used proxy server is the main task at hand.

So, see you in the next post soon with some major leaps of advancement in this direction. Till then, stay connected, keep hacking and listen to Nirvana…

“Think twice, code once”

Fine Stroke: Capturing Traffic

Amidst the much awaited New Years Eve and the sound of ringing jingle bells, still hovering I am much excited to share my next Outreachy update.

So, in the last post I  mentioned about what attracted me to contribute to Electronic Frontier Foundation in particular and how I landed up with this project.There were no technicalities therein but this post is intended for describing what my project exactly is and how far I have come by now.

It has taken exhaustive learning and surfing from my end to be in a situation apt for this write-up, so I will try to pour out as much as I can and I remember going down my memory lane (and my browser history :P) for past few weeks.

As a part of my internship, I am working on developing Privacy Badger for android devices.If you are pondering, what is Privacy Badger, there you go:

” Privacy Badger is a browser add-on that stops advertisers and other third-party trackers from secretly tracking where you go and what pages you look at on the web.  If an advertiser seems to be tracking you across multiple websites without your permission, Privacy Badger automatically blocks that advertiser from loading any more content in your browser.  To the advertiser, it’s like you suddenly disappeared.”

eff_privacy-badger

To learn more about it and check how it differs from other ad-blockers or what principles it runs on follow this link: Learn More 

Essentially, privacy badger is available in form of browser plugins for Firefox and Chrome. We need to provide the same for mobiles too.

The question is how? We can make it happen in any of the three ways:

  1. Making a Firefox add on for Android.
  2. Standalone Android  Browser App.
  3. To use a VPN connection in Android to regulate the traffic across the device.

following from discussion with my mentors.

Given the time, scalability ,resources and efficiency constraints, the best suited approach is the third one. So, I began learning about VPN and how can we make such connection on Android devices, in parallel checking out similar apps like Disconnect, AdBlock Browser, Ghostery etc.

We all have used VPN sometime or the other but when it comes to creating one programmatically you need to go to the core of what exactly constitutes it and how can you build one. Technically, VPN connects securely. It uses “tunneling protocols” ; it sets up tunnel between your system and server through all the routers going through, and encrypts all info going through the tunnel. If any hacker is trying to hack at any router, tunnel detects penetration, then the tunnel shuts down and new path gets created.

Android has classes such as VpnService that supports setting up of VPN connection. After trying out few toy examples I was ready to setup my VPN connection. While I was under the notion that VPN will essentially ensure users’ anonymity while surfing, the intended use was not that.

Further communication clarified this out as to what exactly we are looking for: “The idea is that we would run a vpn server locally on the phone to catch any traffic that is going out and look for domains that appear like they could be tracking and then keep a count of those domains and report them to the user in some sort of dashboard” as Cooper guided me.

The very idea of creating a server on android itself fascinated me 🙂 I read couple of blog posts later deliberating on how you can host your websites successfully on spare Android devices. I checked out and created a server on Android using nanohttpd yet another amazing open source project.

So I had my localhost running on Android. I setup a VPN connection to it but then what next? How to channelize all the outgoing traffic via it? This thing was getting app-specific, as in if I use a particular app and make requests via it, they may get directed to the vpn server and I can then do the intended traffic capturing or filtering, but I wanted to capture the entire outgoing traffic from my device.

After surfing and posting in related communities, I found some really good tools and ways to do that. Being stuck for sometime, meanwhile I tried out Charles Proxy and Firesheep, read more about cookie profiling and browser fingerprinting.

Then I decided why not try this by creating a proxy server on Android, so that all requests are first directed to this server, captured and then to the intended server (forward proxy). So I created a proxy server on Android using the inbuilt support for the same. Since, my android device was connected to Wi-Fi I added the manual proxy settings to it that directed all the requests to this server hosted on the device itself.

After failed attempts to establish the much needed connections and get the GET/POSTs requests captured , I was having a broad smile when it actually finally happened. Different  issues come up while dealing HTTP and HTTPS requests respectively. I will be writing a detailed post on my technical blog explaining the major code snippets for the same.

I got a sneak peek into how WhatsApp uses sockets to communicate, while trying out all this.

As of now, I believe the last few days of year 2015 have been quite enthralling and engaging and the year is ending up with good hope to make things work next.

I will be trying out more possibilities and refine the present architecture, along with working on the app’s features and functionalities.

So, that was more or less the log of events at my end.

Oops, I just checked the post went long a bit, so since you have reached till here, I take the opportunity to wish you a very happy and prosperous New  Year 2016 !

new-year-jokes_o_983387.jpg

See you in next post soon!

Till then stay connected, happy hacking, have your favorite cuisine and if you are a fan of metal covers, do check out this guy:

Lady Gaga goes metal

Adieu.

 

 

 

 

The blank canvas : Beginning

Hola folks!

So now the time comes when I will  officially contribute to FOSS as an Outreachy intern. Yayy!  😀

Before I begin shedding the technicalities of the project in this blog, I would like to log the journey till now and how I ended up taking this project.

It is the incident, a few months back, when the semester began. I remember taking a morning stroll with two of my friends when we were discussing about how the websites bring in the ads/content we search on in the parallel tabs of our browser. You would have noticed, when you search for some “XYZ” content in your browser tabs, the targeted ads and in fact content of your feed changes accordingly. It seems like someone is keeping a track of our online activity very diligently, and the feeling of being tracked feels so bad.

Later that day, while surfing through the Outreachy projects, I stumbled upon Privacy Badger and it immediately captivated my interest. I read about it, forked its github repo, tried understanding the design and implementation and started contributing 🙂

The more I got to know about the organization, Electronic Frontier Foundation and the umbrella of projects it deals with , the more sure I became that this is the one organization I wish to work with.

And so I submitted my Outreachy application, having made the initial commits and PRs. I still remember the happy shiver down my spine, I got when I saw my name in the selected list.

As of now I am understanding it in a more exhaustive manner along with working on couple of bugs, so that I am able to develop a concrete conceptual base for my application to stand upon. So, frankly its the warm-up phase.

In the next post I will be sharing my discoveries by then and the progress made.

Till then,

Have coffee, keep hacking!

And as I go, some good music explored this week: MUSIICCC

See you in next post!