So for the past few months, I’ve been working with food writer Michael Ruhlman on the iPhone adaptation of his cookbook “Ratio”. Well pop some bubbly, cause the app is now live in iTunes!
Purchase Ratio for iPhone and iPod Touch
I’ve been using MobilePress for a while now on my blog. It’s a quick, easy way to make your WordPress blog friendly for all those crazy smartphones out there like iPhone, Android, Nokia, Palm Pre, etc. etc. You just add the plugin to your WordPress install, activate it, and you’re set.
My only issue with the plugin is that it doesn’t support alternate content for mobile devices. Sure you can post the same text in both places, but it doesn’t always work that way for things like very large images or YouTube embeds. Often you’ll want the desktop version to have the very large image, but the mobile version to have a small thumbnail. Likewise with videos you’ll want your desktop site to have the full video embed, but the mobile site to have a link to the 3GP version of your video. To solve this, I quickly wrote an extension to MobilePress that allows you to accomplish this using WordPress shortcodes. Here’s the result.
[mobile]
[/mobile]
[nonmobile]
[/nonmobile]
If you’re on the mobile site, you should only have seen a small thumbnail linking to a mobile version of the video. But if you’re on the normal version of the site, you’ll get the full video embed.
So how did that work exactly? In the post itself, place [mobile] at the beginning and [/mobile] at the end of any text you want to appear only on the mobile site. Likewise, for content that you don’t want to appear on the mobile site, you place [nonmobile] at the beginning and [/nonmobile] at the end of any text. For this post, here’s what the text looked like.
[mobile] <h1>You're on the mobile site!</h1> <a href="rtsp://rtsp-youtube.l.google.com/CkYLENy73wIaPQk4TalGJzJtDRMYDSANFEIJbXYtZ29vZ2xlSARSBXdhdGNoWg5DbGlja1RodW1ibmFpbGCLjNC-9__5m0oM/0/0/0/30/video.3gp"> <img src="http://i.ytimg.com/vi/DW0yJ0apTTg/default.jpg?w=160&h=120&sigh=uxtjuGf9Vjatje_YL3m1JSn0mMY" alt="video" width="160" height="120" style="border:2;margin:0px;" /> </a> [/mobile] [nonmobile] <h1>You're on the non-mobile site!</h1> <object width="425" height="344"> <param name="movie" value="http://www.youtube.com/v/DW0yJ0apTTg&hl=en&fs=1&rel=0"&rt;</param&rt; <param name="allowFullScreen" value="true"></param> <param name="allowscriptaccess" value="always"></param> <embed src="http://www.youtube.com/v/DW0yJ0apTTg&hl=en&fs=1&rel=0" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="425" height="344"></embed> </object> [/nonmobile]
Pretty easy, eh? Here’s the source code for the plugin.
function mobilecontent_func($atts, $content = null) {
if ($_SESSION['SESS_MOBILE_ACTIVE'] === TRUE) {
return $content;
} else {
return "";
}
}
function nonmobilecontent_func($atts, $content = null) {
if ($_SESSION['SESS_MOBILE_ACTIVE'] === TRUE) {
return "";
} else {
return $content;
}
}
add_shortcode('mobile', 'mobilecontent_func');
add_shortcode('nonmobile', 'nonmobilecontent_func');
That’s all there is to it. Each of these functions just checks for the existence of a session variable telling whether or not you are using the mobile version of MobilePress (the cookie management is handled entirely by MobilePress). Based on the session cookie, it either returns the text itself, or it returns an empty string.
I’ve only spent a few minutes writing the plug in, so I am curious if others would find this useful or if there are any obvious bugs or enhancements people notice.
You can download this MobilePress extension here, and let me know what you think.
May 30-31 is BarCampNYC4. Register now as slots are disappearing quickly.
I’m putting together something to talk about RepresentedBy and its future development roadmap. Will anyone else be working on something around open government, or technology for change? Would love to assemble a group session on this.
Recently, I’ve been playing around with Minify. For those who’ve never heard of it, Minify is a simple HTTP content server built as a PHP5 application. It’s placed in a folder on your server, and then either using a querystring or mod_rewrite, it will dynamically merge and minify CSS and JS files as they are served up to a user’s browser. This allows you to develop using multiple CSS or JS files with comments and clear visual formatting, but when they are delivered to your users via a browser, they get minified and optionally merged making your site load and render faster in the user’s browser.
While Minify is meant to run on a webserver, I thought it could also work as a handy offline tool for optimizing Nokia WRT widgets. Since WRT widgets are just zipped folders of Javascript, HTML and CSS, Minify seems like a simple step you could include in the widget packaging process of your workflow before testing and deploying.
I wasn’t sure how much savings Minify would give the average WRT widget, so as a quick test I batch processed the contents of the STEW sample widget from Forum Nokia using Minify running on a local webserver. The resulting files were bundled as a new widget, and here’s a comparison of the two.
| Size of all Files | WGZ size | |
|---|---|---|
| Forum Nokia version | 122.4K | 83.0K |
| Minified version | 106.3K | 78.2K |
| Difference | 13.2% smaller | 5.8% smaller |
The savings aren’t gigantic, but they’re significant enough that you should seriously consider adding this step before deploying a widget. I’d also be curious to see if anyone has done comparisons about whether or not Minify affects how much memory is consumed at run-time in the browser.
I might do some more work on automating this process or creating a tool that minifies widgets automatically for you, but ideally, I think this should be something automatically built into widget IDEs like Aptana.
How are other developers out there currently optimizing their widgets for release on Ovi Store?
I’ve spent the past two days attending the inaugural Nokia Developer Summit. For those interested who weren’t able to attend, there’s been coverage on Nokia Conversations, and on Twitter using the #nds09 tag.
The high level takeaways from the show are:
I don’t attend a lot of conferences, but if I had to give this one a letter grade, I’d give it a solid B.
WHAT WORKED
WHAT DIDN’T WORK
But overall, I thought it was a great experience. I’m curious what others out there are thinking (regardless of if you attended). Comments?
UPDATE: That was fast, just got an email that the widget has been accepted!
I’ve officially submitted the Morse Code widget to the Ovi Store where hopefully it will downloaded onto millions of devices worldwide. And guess what? It’s FREE.
In case you’ve forgotten about it, here’s a brief video showing how it works.
Over on Taxpayers for Common Sense, they’ve been doing a good job of keeping up with how well House members are complying with new rules about posting earmarks and appropriations to their websites.
The webpage on taxpayer.net gives some insight into their process, but unfortunately, their Excel spreadsheet only uses legislator’s names. This is fine for most individual projects, but if you want to mashup the data across multiple data sources, then ideally you need a common identifier like a GovTrack ID or a BioGuide ID.
Anyway, I took a quick stab at writing an Excel macro which attempted to figure out people’s bioguide IDs. Here’s the results:
Database of FY2010 Earmark Requests (with BioGuide IDs, version 8)
Note that this work was done with version 8 of the Taxpayer.net database, so if they continue to release new versions of their spreadsheet, this representation might be a bit behind.
For the past two months, I’ve been developing RepresentedBy, a Facebook application created for the Sunlight Labs Apps For America competition. During the two months of development, I’ve immersed myself in the online government world and while I’ve been exposed to quite a lot of great work by passionate individuals, I’ve also realized how little of Congress is digitally transparent.
Digitally transparent can mean a lot of things to different people, so in an attempt to quantify that, I’ve developed what I’m calling the DTI, or Digital Transparency Index. This is a number between 0 and 115 that gives you a rough idea of how engaged a legislator is in the digital world. Legislators are scored on the following criteria:
The sad truth is that Congress isn’t as digitally immersed as a lot of us. Out of 115 possible points, the highest score anyone received was an 85. Worst of all, out of 451 active legislators, 209 of them scored a big fat zero, 161 legislators scored low (meaning an index of 35 or less), and only 81 legislators scored 40 or higher.
| digital immersion | # of legislators |
|---|---|
| none | 209 |
| low | 161 |
| medium to high | 81 |
My first assumption was that this gap was an age related issue. The average age of Congress is around 60 years old which isn’t exactly the average age of of your cutting edge Internet user. However, I compared the results of the Digital Transparency Index with the number of years that someone has been in Congress and didn’t notice any obvious trends implying a difference based on age. Here’s a graph showing the results.
The far right of this graph indicates highly engaged digital legislators, and the far left of the graph indicates poorly engaged digital legislators. Aside from the large number of legislators who are not digitally engaged, when you start looking closely at highly engaged digital legislators, there’s not a huge disparity between the number of new, younger legislators engaging digitally and older, veteran legislators engaging digitally.
Next, I wondered if there was a connection between digital transparency and earmarks. Taxpayer.net recently released information about active legislators and the earmarks they have included in the 2009 stimulus package so I compared the amount of solo earmarks included by each legislator with their Digital Transparency Index, and graphed the results:
While there is a disproportionately large number of legislators who are not digitally engaged and who have not sponsored large earmarks, you’ll notice that as digital engagement increases, there becomes fewer and fewer legislators who are sponsoring extremely large earmarks. The only exception to this rule is Nancy Pelosi who has a very large Digital Transparency Index (80), but has also sponsored a large number of solo earmarks ($15,667,000).
Is this a trend? Does being digitally engaged and having real-time communication with your constituents discourage legislators from sponsoring earmarks? Or is it the opposite and legislators who don’t support earmarks on principle are more likely to take that message directly to the people and engage with them digitally?
Here’s a table summarizing my findings.
| digital immersion | # of legislators |
avg solo earmarks 2009 |
avg years in Congress |
|---|---|---|---|
| none | 209 | $5,226,898 | 15.8 |
| low | 161 | $6,366,649 | 16.1 |
| medium to high | 81 | $4,069,291 | 15.1 |
If you want to see where your legislator falls on either of these graphs, then check out RepresentedBy, a Facebook application I’m creating which includes this information and personalizes it to your specific district.
Finally, I hope to develop the Digital Transparency Index some more, so if you have any comments or suggestions on how to improve it, then please include them in the comments.
My last entry about Google AppEngine and Facebook Applications was written during the development of RepresentedBy, a Facebook application which is still in a rough beta state, but has finally been opened up to the general public.
The goal of RepresentedBy is to:
The app is still in beta, but once it’s ready for release sometime in late March, then the source code will be made available. In the meantime, please check out the application and let me know what bugs you find, and any problems that you encounter.
For the past six weeks, I’ve spent some of my spare time learning about Python, Google AppEngine and how to create Facebook applications with them. In a few weeks, I’ve learned a thing or two the hard way and thought I would share some lessons learned to save other developers from beginner’s frustration.
1. Never exceed 1000
When working with AppEngine, it’s good practice never to exceed 1000 in anything you’re doing. You name it, this rule applies. For example:
Before you choose to build an app with AppEngine, make sure you can accomplish what you want to do within these limitations. It might make sense to only use AppEngine for part of the whole project (e.g. AppEngine for processing and Amazon S3 for storage).
(**UPDATE** Google recently upped some of these AppEngine limits, but not for everything)
2. AppEngine forces your code to scale out, not up
When I first heard about cloud computing and scalable infrastructures, I thought it meant giant supercomputing clusters which can handle massive amounts of processing and calculations.
AppEngine isn’t like this at all. It’s designed from the ground up to be scalable, but it achieves this by doing hundreds of thousands of small tasks instead of tens of really big tasks. And your source code needs to embrace this philosophy. If your script needs to spend time processing thousands of records, you should re-think why it has to be one script instead of ten smaller ones.
Switching my brain to architect for AppEngine was the hardest part of AppEngine development, but once I got into the groove, it makes total sense. I’ve really enjoyed building my applications from the ground up with scalability in mind. I might not be as open minded if I had to port and existing application to AppEngine, but luckily, I haven’t had to do that yet. 
3. Use DynDNS to develop AppEngine/Facebook apps locally
AppEngine imposes a daily quota of 250 deployments to their server. This limit seems reasonable, but often you’ll need to test your Facebook applications in Facebook itself. And if you’re tweaking CSS or troubleshooting bugs, then you can use up your quota quickly if you have to deploy a new app each time you want to test a change in Facebook. If you use up all your deployments for the day, then you can’t upload anymore and have to stop development until the quota resets in 24 hours.
This has happened to me twice now, and after the second time, I found a great thread in the Developing for Facebook + Google App Engine group describing a solution for using DynDNS or similar service to give a domain name to your local PC, then pointing your Facebook app at your local computer. That way you can test the application on Facebook.com using your local AppEngine devserver. Trust me, this is worth the setup time.
4. Be prepared to dig in, tweak and modify Python libraries
There’s a lot of great Python code libraries out there, but much of it doesn’t work with AppEngine because of AppEngine’s unique webapp framework. You can get most libraries to work with AppEngine by adding a line or two of custom code, but you have to be willing to dig into the code and fix it.
For instance, I’m using the Google YouTube API, and in order for it work with AppEngine, you need to override the http_request_handler like this:
import gdata.service import gdata.urlfetch gdata.service.http_request_handler = gdata.urlfetch
Another example is custom template tags. You need to register your custom tags with AppEngine’s framework:
register = webapp.template.create_template_register()
And then in each of your individual scripts you need to register the library. So for a library named ‘customtags’ it would be:
webapp.template.register_template_library('customfilters')
w00kie has a good blog entry talking about this in more detail, but don’t expect a lot of existing Python libraries to be completely plug-n-play with AppEngine.
5. There is never too much error detection
When a user visits your URL on Facebook, Facebook will call the URL on AppEngine, AppEngine will use its framework to get data from the Internet, from its DataStore, and from Memcache, then return the result to Facebook which processes the FBML and displays the content to the user.
Unfortunately, just about anything can go wrong. I’ve had Facebook authentication fail even though you’re logged in, I’ve had Facebook give up on waiting for AppEngine to render its page, I’ve had AppEngine throw errors when doing a simple urlfetch, and I’ve had third party APIs suddenly stop responding. These errors are rare and normally not reproducible, but you still don’t want your user trying to figure out what an "ApplicationError 5" means. , So write your code to handle lots of exceptions.
6. FBJS is your friend and is key to achieving scalability in Facebook apps on AppEngine
The home page of my Facebook app is a beast. The content you see on the home page comes from more than 30 URLs on 10 different domains and third party APIs. Waiting for AppEngine to download and render this content takes forever, but I was able to pull it off by breaking up the page into five separate pieces. There’s a shell page, and then within that shell page there are four modules which each use FBJS to make a separate AJAX call to AppEngine to retrieve and display their own content.
I’ve learned the hard way that putting all your code in one page can take forever to render and consume lots of CPU, and FBJS helps reduce spread the page load out across multiple scripts.
7. Debugging FBJS is a real pain
While FBJS helps you scale out, debugging FBJS is a real pain. First, it only warns you of syntax errors, so if you have a logical error your script fails without warning. Facebook doesn’t report errors to the browser or allow you to use alerts, so the only solution I’ve found so far is to comment out your JS code one line at a time until you find the trouble spots. I would only advise doing this if you’re developing locally, otherwise you’ll quickly run into your quota limit for daily uploads to AppEngine.
8. If you’re retrieving external content, memcache is your best friend
As I mentioned earlier, the home page of my Facebook app gets most of its content from external URLs. For each URL, you fetch its content, process it into a native Python object (list or dict), and then render the content out via a template. This can eat up your CPU hours, reduce your response time, and make users give up on you.
Using memcache fixes all this. Memcache can store native Python objects, so once you’ve parsed a URL’s content in a native format, store the native object directly in memcache and retrieve it next time a user needs content from that URL
9. Use cron jobs to keep memcache current
Using memcache speeds up response time for all users except your first user. Since you shouldn’t be treating your first user any differently than the others, it’s worth setting up a script that keeps memcache refreshed with external content. This way all users will benefit from the speedup of memcache.
In my Facebook application, I’m retrieving content from a pool of around 3,000 different URLs, so I have set up a script that randomly picks 3-5 of these URLs, retrieves their content, and stores the result in memcache. I’ve also setup a cron job to call this caching script every minute or so and it’s sped up the average response time of the page, because the server never has to go out and retrieve 30 URLs of content at once. Also, if you are using third party APIs that put a limit on your usage, this is a great way to ensure you stay under those limits.
Right now, I’m executing the cron job from my own webserver, but AppEngine has said that cron support is on their roadmap, so hopefully in the future you’ll be able to support this entirely from within your AppEngine setup.
10. Once you’ve built an app or two with AppEngine, you’ll either love it or hate it.
I’ve really enjoyed developing apps with AppEngine, but I will admit it’s not for everyone. Anyone needing to do a lot of heavy data processing, or handle incredibly large data sets will experience nothing but frustration with AppEngine. However, for the majority of online projects, it’s a great way to build something scalable quickly, making it ideal for Facebook applications.
My first major Facebook application should be ready for public beta in the next week or two, so I’ll keep you posted about its progress.