Tom Gersic's Blog
With all the recent buzz around Windows 8 RT for ARM and the Metro interface, I wanted to do a bit of a first-look at developing for the Windows 8 Metro interface. To get started, I thought I’d take a look at getting the Windows 8 Consumer Preview installed on an actual tablet, so I picked up this Acer Iconica Tab, which comes with Windows 7 preinstalled. Getting Windows 8 installed on it wasn’t too hard… You’ll need to burn the Windows 8 .iso file to a DVD, and then plug in an external DVD drive. You’ll then need to enter BIOS setup to alter the boot order so that it finds the USB DVD drive:

Somewhat surprisingly, this entire BIOS setup screen is touch enabled, despite looking pretty much exactly the same as it looked on my 33MHz 486DX back in the early 90s. From there on out, setting up Windows 8 on the device goes quite a bit smoother than setting up the Windows 7 install that comes with the thing (that processes locked up a few times before I got it set up)…

After that whole process, you’ll eventually be left with your very own Windows 8 tablet with the fancy new Metro interface:

It’s not an ARM chip, so the actual device is pretty big and clunky compared to an iPad, but performance is great. One of the interesting things to note is that on Windows 8, the Metro interface basically replaces the start menu. Who knows what Microsoft is planning for the ARM “RT” version of Windows, but at least on the Intel version, there is still a regular Windows desktop. The Start menu button is gone and replaced with a hardware button on the tablet (you can see it glowing in the screenshot above). Some apps, like IE10, have both a Desktop version and a Metro version:

What is Metro, anyway, and why should I care?
Metro is the UI (“design language”) that originated with Windows Phone 7, and is a big part of Windows 8 as Microsoft prepares to challenge Apple and Google in the tablet market. In the consumer market, the Kindle Fire has gobbled up over 50% of the Android tablet market share, which is hovering around 50% of the overall tablet market, but if you’re interested in the enterprise space at all, second place in that market behind the iPad is wide open territory:

Android has been plagued in the enterprise space by the security concerns of Gingerbread (Encryption? What’s that?), lackluster adoption of the more secure Ice Cream Sandwich (ICS) by device manufacturers, and nervousness over fragmentation. iPad has done exceptionally well, with built-in security features that rival the most hardened laptop, but knowing corporate IT’s preferences for Microsoft products, Windows 8 tablets could easily be poised to trounce Android and even challenge Apple in the years ahead.
At the same time, users want apps, and developers want users. Microsoft seems to have recognized that in order to grow the user base, they need more apps being developed, and there aren’t enough users around yet to entice the development community, so Microsoft is paying developers between $60k and $600k to develop for the Microsoft Store that, at 70k apps is lagging behind Android’s roughly 400k apps and Apple’s 600k apps. So, now’s a good time to work on a Windows Metro app.
Metro Design Principles
One of the nicest things about Metro is that Microsoft seems to have taken a cue from Apple’s book (the venerable iOS HIG), and have put some real thought into the UI and these design principles:
Content Before Chrome
Chrome is basically all of the stuff in your app that isn’t content. The toolbars, the tabs, button gradients, scrollbars, etc. One of the core concepts in Metro is to de-emphasize the app chrome and to use the size, weight, and color of typography to convey hierarchical information on the page. Looking at some popular apps, the difference is pretty easy to see in the lack of tab-bars, toolbars, and other application chrome elements. This is Spotify:

Apps are designed to flow from one screen to another instead of having tabs, as is you can see in these Spotify screenshots:

Be Fast and Fluid — Touch First
As with most tablet interfaces, touch gestures are important. Most of these are similar to iOS and Android, but Windows 8 has defined a few new ones, with app commands and system commands being shown in hidden toolbars on the bottom and right edge of the screen.

Be Fast and Fluid — Use Animation Library to Illustrate Common Scenarios
Again, nothing too groundbreaking here, except there are a number of nice animations built into the stock components. Some of them are demonstrated in these demo projects that are worth a look:
Snap and Scale Beautifully
Since Windows 8 apps will run on phones, tablets, netbooks, notebooks, and 30” displays, it makes sense to display more information on larger screen sizes rather than just making things bigger. Just like with the Mobile Web, native apps should handle different screen sizes elegantly, and adapt accordingly.
Snapping is an interesting feature that allows multiple apps to be shown on the screen at the same time. Since Metro isn’t really windowed, this is a way to keep your app on-screen in a sidebar while someone is off doing something else. For instance, if you wanted to monitor Twitter while browsing the maps application:

Use the Right Contracts
Contracts control how Metro apps interact with each other and with the OS.
- Share Sources and Targets allow for sharing between apps.
- Search allows users to search within an app from anywhere in Windows.
- And the File Picker allows users to open or save content between apps in a controlled manner.
Invest in a Great Tile
Tiles are the icons of Windows Metro. Their real power lies in the ability to use the Microsoft Push Notification Server to push out content updates to Live Tiles. Personalized, regionalized, and/or customized content helps keep users coming back to your app.

Roam to the Cloud
Similar to Apple iCloud and Google Drive, Metro apps get per-user cloud storage for saving state to allow users to continue tasks across devices.
Developing Windows 8
Getting started is easy and free. Just download the Windows 8 Consumer Preview and Visual Studio 11 Express Beta. Both can be installed in a VM, on a desktop/laptop, or on an actual tablet. From there, you basically just choose a template to start with. Your choices are Javascript, VB or C# .net, and C++ (mostly useful for DirectX games).

The Native options give you a handy XAML (eXtensible Application Markup Language) visual designer:

The Javascript templates, though, are really interesting because they give you the ability to create an app entirely in HTML5/Javascript, similar to what you would do on other platforms with PhoneGap/Cordova, but without the need for any external libraries.
So, that's Metro. What are you doing with it?
Storing Data Offline with Salesforce Mobile SDK SmartStore
Say you’re writing a “hybrid” mobile app for iOS and Android using PhoneGap (“Callback” or “Cordova”) and you want to store data locally. Your choices are kind of limited. You could, for instance, use WebSQL, which has been implemented in WebKit for a while now and is available on both iOS and Android devices. If you clicked that link, though, you’ll probably notice the big scary warning that WebSQL has been deprecated…

It works right now, but maybe it’s not the best idea to use it if you don’t want to have to strip it out someday down the road. You also have to deal with all the data size limitations and requests to the user to allow the app to increase the database size.
So, what’s up next from the W3C if the WebSQL spec isn’t being maintained? The new spec is IndexedDB, which is a “database of records holding simple values and hierarchical objects”. Perfect! We’ll just check caniuse.com to make sure it’s supported on iOS and Android:

So… okay. It looks like it’s not supported yet on either. Does PhoneGap provide anything? In fact, it does, the Storage class. Which is based on WebSQL… I think you can see where I’m going here.
What do you do? Well, the Salesforce Mobile SDK for iOS and Android has implemented an ORM layer called SmartStore abstraction to SQLite, which uses FMDB on iOS and SQLCipher on Android. It’s easy to use and allows you to store data in JSON format locally and specify indexes for searching and sorting. Also, it uses native implementations of SQLite which are accessed via PhoneGap plugins so you don’t have any issues with HTML5 storage limitations or the W3C changing their mind about what web database technology everyone should standardize on.
To demonstrate SmartStore, I’ve put together a simple demo app, and posted it to Github:
https://github.com/tomgersic/SmartStoreDemo
The Sample App
Here it is running on the iPad simulator. One thing to know about SmartStore is that each database Table is called a Soup. It’s an Apple Newton thing. The Newton’s file system was based on a “Soup” store model. So here, I’ve asked the app to create a new Soup named “EggDrop” specified that I want to use just the “id” field as an index, and entered a Salesforce REST API query to be executed using the handy ForceTK library.

To start off, download the code from GitHub and open up the XCode project. If you want to start from scratch, you’ll want to install the Salesforce Mobile SDK for iOS and selected “Hybrid Force.com App” from X-Code’s New Project wizard.
Going from top to bottom:
- Soup Name: A “Soup” named this will be created.
- Indexes: I’ll show how this gets represented in the data model in a bit, but you can specify JSON fields for indexing and specify a type of either “string” or “integer”.
- REST Query: Basically any query to the Salesforce REST API. Here we’re just defaulting to pulling down my Chatter feed items. I have 3 in my feed in a dev org that we’ll pull down.
- Query SFDC: Run the REST Query, save the results to the specified Soup, and log the results to the console (on the page).
- Query SmartStore: Query the specified Soup and log the results to the console. They should be the same results as the remote Salesforce Query.
- Reset: Delete all the Soups that we’ve created during this session.
- Logout: Log out of your Salesforce org.
SmartStore Soup Functions
To see how all this works, the interesting bits are in the inline.js file.
- To register a Soup, we call registerSoup:
navigator.smartstore.registerSoup(soupName,indexes,success,error);
- To add some records to that Soup, we call upsertSoupEntries:
navigator.smartstore.upsertSoupEntries(soupName,records, success, error);
- To query the Soup, first we have to define a querySpec (buildAllQuerySpec just pulls all records rather than searching for specific records):
var querySpec = navigator.smartstore.buildAllQuerySpec("id", null, 20);
This querySpec specifies “id’ as an index, null (allow default) for the sort order, and 20 for the page size.
- We then use that querySpec to get records back from the Soup:
navigator.smartstore.querySoup(soupName,querySpec, success(cursor),logError);
- Additionally, if we want, we can delete Soups with removeSoup:
navigator.smartstore.removeSoup(record,success,error);
Running the App
When you run the app for the first time, you’ll be presented with an oAuth 2.0 User-Agent login screen for Salesforce. If you don’t already have a Developer Org, get one at http://developer.force.com/. It’s free.
Once you log in, hit Query SFDC, and you see the Soup get registered, JSON data gets downloaded from the REST API and added to the SOUP. Take a look at the JSON data, and then hit the “Query SmartStore” button. You’ll see it load that same data from the SmartStore Soup and display it in the console log.
The Database
Let’s take a look at the actual SQLite database file and see how things are stored behind the scenes. You’ll see we have 3 tables in the database: soup_names, soup_index_map, and TABLE_1 (sqlite_master and sqlite_sequence are used internally by SQLite):
- soup_names keeps a list of the Soups I’ve created

- soup_index_map keeps track of the indexes for each Soup.

- TABLE_1 is a generically named table that contains my JSON data and Index values. The TABLE_1_0 field here is the “id” index we specified when we created the Soup. If I had created more indexes, like a “name” index, another column would be shown here.

The Soup field contains the full JSON response for that record so you can reuse whatever code you wrote to deal with the JSON record from the REST API once you’ve pulled it from the Soup.

So that’s SmartStore. It’s a great way to put together an app for Salesforce, Database.com, or Chatter that stores JSON data on the device for offline access.
One of the things added to Safari and UIWebView in iOS5 is support for XMLHttpRequest 2, which according to W3C adds new features "such as cross-origin requests, progress events, and the handling of byte streams for both sending and receiving". As part of the last bit of that, it's now possible to set the responseType to arraybuffer, which is "used to represent a generic, fixed-length binary data buffer". More info here. This is useful if you want to download binary such as image or audio data from a remote location, and potentially manipulate it before presenting it to the user.
This has many possible applications for file transfer, but for instance, if you're writing an HTML5 / PhoneGap app for iOS 5, and you want to download an attached file from the current version (v23.0) of the Chatter REST API, you need to send a GET request to:
https://test.salesforce.com/services/data/v23.0/chatter/files/[PUT THE FILE ID HERE]/content?versionNumber=1
But, you have to include a few headers for authentication…
setRequestHeader("Authorization", "OAuth " + oauthToken);
setRequestHeader('X-User-Agent', 'salesforce-toolkit-rest-javascript/v23.0');
…so it's not like you can just dump the GET request into the src of an <img> tag. So, what you can do now in iOS 5 is send off an XMLHttpRequest with a responseType of "arraybuffer", and use this data to write to a HTML5 Canvas object.
Here's the XMLHttpRequest:
var request = new XMLHttpRequest();
request.open("GET", url, true);
request.responseType = "arraybuffer";
request.setRequestHeader(that.authzHeader, "OAuth " + that.sessionId);
request.setRequestHeader('X-User-Agent', 'salesforce-toolkit-rest-javascript/' + that.apiVersion);
request.onreadystatechange = function() {
// continue if the process is completed
if (request.readyState == 4) {
// continue only if HTTP status is "OK"
if (request.status == 200) {
try {
// retrieve the response
callback(request.response);
}
catch(e) {
// display error message
alert("Error reading the response: " + e.toString());
}
}
}
}
request.send();
And here's the body of the callback that handles Base64 encoding the arraybuffer and writing that data to a Canvas using the Data URI Scheme:
function(response){
var imageCanvas = $('#'+value.id);
var cxt = imageCanvas[0].getContext("2d");
var myImage = new Image();
myImage.src = "data:"+value.mimeType+";base64,"+base64ArrayBuffer(response);
imageCanvas[0].width=myImage.width;
imageCanvas[0].height=myImage.height;
cxt.drawImage(myImage,0,0,myImage.width,myImage.height);
}
For the Base64 encoding, I used this handy function that I found
over on PasteBin.
Note, I haven't tested it, but this should (
probably) work on Android OS 3.0 (Honeycomb) as well.
If you're interested in doing all this with
ForceTK, take a look at this
Pull Request over on GitHub.
Dreamforcenger Hunt is an Android GPS based scavenger hunt for Dreamforce 2011. You can play during the week of Dreamforce, and win a great Samsung Galaxy Tab 10.1. The game is open now through Noon, PST, on Friday, September 2. The app uses Database.com, oAuth 2.0, the SFDC REST API, and a custom Apex Webservice. We will be digging through the code in the Application Development with Android session, so I hope to see you there.
The code for this app will be released on Github.
To play, you must first register at http://bit.ly/dreamforcenger.
You can then download and log into the Dreamforcenger Hunt app, which is available in the Android Marketplace:
Rules of the Game
1. Dreamforcenger Hunt is played by registering and installing the Dreamforcenger Hunt app on your Android phone. This app is available in the Android Market. In the app, you will find a list of clues for scavenger hunt locations around San Francisco. To find a location, you will have to go to that location, and click the "Geotag It!" button. If you have found the item, the app will tell you so, and it will be marked as found in Database.com for your user.
2. The person that finds the most scavenger hunt items will win the prize, a Samsung Galaxy Tab 10.1
3. To be eligible to win, you don't need to attend the Application Development with Android session, but you do need to be a Dreamforce 2011 attendee.
4. Employees of Model Metrics and Salesforce.com may participate in the scavenger hunt, but are ineligible to win the prize.
5. In the event of a tie, the winner will be chosen at random from among the tied users.
6. Additional hunt locations may be added after the game begins, so if you find all of them, don't assume more won't be added.
7. The game ends at Noon, PST, on Friday, September 2. A winner will be chosen at that time. The winner may pick the prize up in person, or if that is impossible, the prize will be mailed to the winner at a later date.
8. You must register with your real name and email address. Accounts that use obviously fake names or email addresses will be deactivated.
9. Any attempts to hack the system will result in an ineligibility to win the prize. Please play fair.
10. Have fun!
At this point in time, we've implemented the oAuth 2.0 User-Agent flow and the Refresh Token flow for iOS, Android, and Flex/AS3. I figure that makes us as much an expert at doing this as anybody, so I thought I'd take a moment to describe some of the details. First off, the reason you want to use oAuth 2.0 when developing apps for mobile devices… no token. We've been developing mobile apps for Salesforce.com for the last 4 or so years, and the need to provide a username, password, and token has always been a pain point. Since it's a 24 character alpha-numeric string, this was especially problematic back before iPhones had copy/paste functionality ("is that a l, an I or a 1?"). With oAuth 2.0, you can finally get rid of having to worry about the token.
oAuth 2.0 is a popular universal specification for authentication to various web services. If you've used a mobile app that logs into Facebook, Twitter, LinkedIn, or Chatter, you've probably used it. for Salesforce.com provides four different authentication flows:
- Web Server
- User-Agent
- Refresh Token
- Username/Password
A combination of the User-Agent flow and Refresh Token flow are recommended for mobile applications, so that's what I'll demonstrate here.
First off, you should understand both flows from a high level:

Salesforce Configuration
Both the User-Agent flow and the Refresh Token flow require a Remote Access Application be set up in the target SFDC org. This is configured under Setup=>Develop=>Remote Access. Required fields are Application, Contact Email, and Callback URL. There are a variety of rules about what the Callback URL can be, but the simplest way to do this is to have it be: https://login.salesforce.com/services/oauth2/success
Once saved, SFDC will generate and display a Consumer Key and a Consumer Secret. Both of these will be needed by the application for login.

User-Agent Flow
The User-Agent flow involves the use of a webview within the application. The app passes a special Salesforce.com URL to that webview, which renders a login view.
First, the user will be asked to log in, and then they will be asked to confirm that they would like to provide access to Salesforce.com using this application:

The URL passed to SFDC in order to render this login view is in this format:
https://login.salesforce.com/services/oauth2/authorize?
response_type=token&
display=touch&
client_id=[CONSUMER KEY FROM REMOTE ACCESS]&
redirect_uri=https%3A%2F%2Flogin.salesforce.com%2Fservices%2Foauth2%2Fsuccess
Upon successful login, SFDC will redirect the webview to the URL specified as the redirect_uri (which must be the same as the Callback URL specified in the Remote Access Application setup). After the Callback URL will be a hash tag and then a series of parameters returned by Salesforce:
access_token=[ACCESS TOKEN (Session Id)]
&refresh_token=[REFRESH TOKEN]
&instance_url=https%3A%2F%2Fna1.salesforce.com
&id=https%3A%2F%2Flogin.salesforce.com%2Fid%[ORG ID]%[USER ID]
&issued_at=1312403866216
&signature=[SIGNATURE]
The Access Token specified here is the Session ID that will be used for all subsequent calls to the API. The Refresh Token must be saved securely to disk, as it will be used in conjunction with the Consumer Key and Consumer Secret to get a new Access Token from SFDC when the current on expires. The Org Id, User Id, Issued At Time (number of milliseconds since the Unix Epoch), and Signature should be saved as well.
Refresh Token Flow
At some point in time, your Access Token will expire. This may come as a shock, so be sure to prepare your friends and family. The app will learn that the session ID has expired when it attempts to access the API and the response is either:
- SOAP API: HTTP 500 Internal Server Error, with a faultCode: <faultcode>sf:INVALID_SESSION_ID</faultcode>
- REST API: HTTP 401 Unauthorized

The amount of time a session ID remains valid is configured under Security Controls => Session Settings in SFDC Setup. When it expires, the app will have to use the Refresh Token flow to request another Access Token from SFDC. To do this, the application will send a POST request to SFDC including the Refresh Token, the Consumer Key, and the Consumer Secret. SFDC will respond with a new Access Token.
NOTE: At no time does the application store the Username or Password of the individual logging into the app.

So, that's it. I hope you've enjoyed this foray into the world of oAuth 2.0 and Salesforce.com.
There are a few different ways to deploy metadata from org to org with Salesforce.com. The three main options are to use Eclipse, to use Ant (the "Force.com Migration Tool"), or to use Change Sets. The first two are completely manual to set up (although Ant, obviously, is able to be run over and over again). Change Sets have a lot of promise, because they do handy things like searching for dependencies, but as of this writing, they are still prone to missing important bits, especially with profiles, so you can't rely on them to produce a perfect deploy from one org to another. Consequently, it's important to be able to quickly validate that a deployment was successful, and that everything that you meant to deploy from one org to another actually did get deployed.
Enter DiffDog…
DiffDog is a great tool for validating that the metadata between two orgs is identical, and, when used in conjunction with Eclipse, it can be used to push changes from one org to another. It can be used to compare any of the metadata types that are able to be checked out using Eclipse: Objects, Page Layouts, Profiles, Workflow, Reports, etc. The main benefit with this tool over other diff tools is that it allows for the comparison of XML files with ignoring the order of XML nodes. This is important because the metadata between two orgs is XML based, and can be functionally identical, but rendered in different orders. Because of this, with a regular flat-file diff tool, you will get lots of false positives. DiffDog can be configured to properly compare XML files, thus eliminating these false positives. This post describes some optimal settings for use with SFDC, and the process for comparing orgs and deploying changes.
To start with, download the tool from Altova:
http://www.altova.com/download/diffdog/diff_merge_tool_professional.html
Setup
Once you've downloaded the app and registered it, go to the Tools menu, choose "Comparison Options", and select the XML tab. This part is important: you want to make sure the comparison ignores the order of child nodes. This basically means that XML nodes can be rendered in any order and still be considered identical. Click the "Ignore order of child nodes" box in the Order section. All of the other options should be default, but double-check to make sure they match this screenshot.

Additionally, if "quick" diff is turned on for folder comparison (it will be by default), make sure to turn it off:

You want to do extension-based comparison (EXT):

Check Out the Orgs
You will now have to check out the metadata objects that you want to compare from both SFDC orgs using Eclipse. Let's assume one org is a sandbox and one is production. Note that if you want to compare profiles, you will need to select all of the metadata types for everything that you want to compare profile permissions for. For instance, if you want to compare Field-Level Security on Custom Objects, you will need to check out Profile metadata AND Custom Object metadata. SFDC only sends the profile metadata for the metadata types that you have checked out. If you try to do only profiles, the files will be practically empty.

The metadata from the two orgs that you checked out will be located in your Workspace directory. You can generally figure out where this is by right-clicking on one of the files in Eclipse, and selecting "Properties".
Using DiffDog
You'll then want to pick a metadata type and open the two org's folders for that type in DiffDog. For instance, to compare Objects, select "Compare Directories" from the File Menu…

…and then select a metadata folder for each org. I'd suggest putting your sandbox org on the left and your production org on the right so that you're moving changes from left to right, but you can do it either way.


Once you've done this, DiffDog will initiate a high-level diff of all files in the directories, and will display something like this. Lines displayed in black are identical, lines displayed in Red have differences, and lines in blue are missing in one org or the other.

If you need to move an entire object over, you can do that here by clicking on the blue name and pressing the "Copy from left to right" or "Copy from right to left" button, depending on which direction you want to go.

If you want to inspect the differences between two files, double-click on one, and it will launch a flat file-based diff that looks something like this:

This is not what you want. You want to select the "Grid View" tab at the bottom left of the window. This launches the grid-based diff tool that will show you differences between the two metadata objects:

Differences are highlighted in light green, and the "current difference" is highlighted in a darker green. To move a change from one org to another, you'll have to click on the box in the grid in the org you want to move FROM, and then click the "Make current difference" button in the top toolbar (or hit Alt-Enter) to highlight it in dark green.

Once you've done this you can copy the change over with the "Copy from Left to Right" button:

Deploying Your Changes
You'll then want to save (Ctrl-S or File=>Save). This will save your changes locally. Note that they have not yet been deployed to SFDC. To do this, you'll have to go back to Eclipse. Find the file (or group of files) that you saved in your Eclipse project, right click on it, and select "Refresh". This will cause Eclipse to attempt to deploy your changes to SFDC. This could result in one or more errors, so be sure to watch the Problems tab for any errors. If you're deploying to Production, this step can take some time if the org has a lot of Apex code, because all tests will be re-run when you deploy. A minute or so is common. 10-15 minutes isn't unheard of.

The well-publisized outage of EBS on multiple availability zones in the US-EAST-1 Region of AWS last week kicked off some excellent blog posts from companies who, through robust architectural choices, managed to weather the storm quite well. It lasted five days, it's been called the worst cloud computing disaster ever, and Amazon's communications strategy didn't exactly shine, but it has presented an opportunity to learn from the companies that are hosting their sites on the AWS cloud better than many of their peers.
This is just a round-up of some of these posts, and the advice given. They've been edited down, of course, so be sure to read each of these articles for the whole story:
The Cloud is Not a Silver Bullet – Joe Stump, CTO of SimpleGeo
- Everything needs to be automated. Spinning up new instances, expanding your clusters, backups, restoring from backups, metrics, monitoring, configurations, deployments, etc. should all be automated.
- You must build share-nothing services that span AZs at a minimum. Preferably your services should span regions as well, which is technically more difficult to implement, but will increase your availability by an order of magnitude.
- An avoidance of relying on ACID services. It’s not that you can’t run MySQL, PostgreSQL, etc. on the cloud, but the ephemeral and distributed nature of the cloud make this a much more difficult feature to sustain.
- Data must be replicated across multiple types of storage. If you run MySQL on top of RDS, you should be replicating to slaves on EBS, RDS multi-AZ slaves, ephemeral drives, etc. Additionally, snapshots and backups should span regions. This allows entire components to disappear and you to either continue to operate or restore quickly even if a major AWS service is down.
- Application-level replication strategies. To truly go multi-region, or to span across cloud services, you’ll very likely have to build replication strategies into your application rather than relying those inherent in your storage systems.
How SmugMug survived the Amazonpocalypse – Don MacAskill, CEO of SmugMug
- Spread across as many AZs as you can. Use all four.
- If your stuff is truly mission critical (banking, government, health, serious money maker, etc), spread across as many Regions as you can.
- Beyond mission critical? Spread across many providers.
- Since spreading across multiple Regions and providers adds crazy amounts of extra complexity, and complex systems tend to be less stable, you could be shooting yourself in the foot unless you really know what you’re doing.
- Build for failure. Each component (EC2 instance, etc) should be able to die without affecting the whole system as much as possible.
- Understand your components and how they fail. Use any component, such as EBS, only if you fully understand it. For mission-critical data using EBS, that means RAID1/5/6/10/etc locally, and some sort of replication or mirroring across AZs, with some sort of mechanism to get eventually consistent and/or re-instantiate after failure events.
- Try to componentize your system. Why take the entire thing offline if only a small portion is affected?
- Test your components. I regularly kill off stuff on EC2 just to see what’ll happen.
AWS outage timeline & downtimes by recovery strategy – Eric Kidd, Randomhacks.net
Eric took an interesting look at various potential strategies, and how long a company would have been offline during the EBS outage:
- Rely on a single EBS volume with no snapshots: 3.5 days
- Deploy into a single availability zone, with EBS snapshots: over 12 hours
- Rely on multi-AZ RDS databases to fail over to another availability zone: longer than 14 hours for some users.
- Run in 3 AZs, at no more than 60% capacity in each: This is the approach taken by Netflix, which sailed through this outage without no known downtime
- Replicate data to another AWS region or cloud provider: This is still the gold standard for sites which require high uptime guarantees.
The AWS Outage: The Cloud's Shining Moment – George Reese, Founder of Valtira and enStratus
The Amazon model is the "design for failure" model. Under the "design for failure" model, combinations of your software and management tools take responsibility for application availability. The actual infrastructure availability is entirely irrelevant to your application availability. 100% uptime should be achievable even when your cloud provider has a massive, data-center-wide outage…
There are several requirements for "design for failure":
- Each application component must be deployed across redundant cloud components, ideally with minimal or no common points of failure
- Each application component must make no assumptions about the underlying infrastructure—it must be able to adapt to changes in the infrastructure without downtime
- Each application component should be partition tolerant—in other words, it should be able to survive network latency (or loss of communication) among the nodes that support that component
- Automation tools must be in place to orchestrate application responses to failures or other changes in the infrastructure (full disclosure, I am CTO of a company that sells such automation tools, enStratus)
Today’s EC2 / EBS Outage: Lessons learned – Stephen Nelson-Smith, Technical Director of Atalanta Systems
- Expect downtime…What matters is how you respond to downtime
- Use amazon’s built-in availability mechanisms
- Think about your use of EBS:
- EBS is not a SAN
- EBS is multi-tenant…Consider using lots of volumes and building up your own RAID 10 or RAID 6 from EBS volumes.
- Don’t use EBS snapshots as a backup…Although they are available to different availabilty zones in a given region, you can’t move them between regions.
- Consider not using EBS at all
- Consider building towards a vendor-neutral architecture…Cloud abstraction tools like Fog, and configuration management frameworks such as Chef make the task easier.
- Have a DR plan, and practice it
- Infrastructure as code is hugely relevant…one of the great enablers of the infrastructure as code paradigm is the ability to rebuild the business from nothing more than a source code repository, some new compute resource (virtual or physical) and an application data backup.
Okay, so the title is a bit harsh.
I was intrigued by the rather excellent post over at the blog Il y a du thé renversé au bord de la table, [Rant] Web development is just broken. Yoric makes the argument that web developers are forced to deal with too many "nightmares" that have very little to do with programming. First you have to decide on a programming language. Should you use PHP, C#, Java, Ruby, Perl, or Python? Then you have to choose a web server and OS. Windows/IIS or *nix and Apache? OSX? BSD? Solaris? If you go with Linux, which distro do you choose? Is it worth it to pay for Red Hat, or will Fedora do? What about Ubuntu? Then you have to choose a DBMS, of course. Do you want Oracle? Well, can you afford Oracle? Then there's MySQL, SQLServer, or PostgreSQL. Or maybe one of the NoSQL databases like MongoDB, CouchDB, or Cassandra. And then you probably want to choose a server-side framework. Rails? Spring? Zend? And a client-side framework, of course, so you don't have to worry too much about all the differences between the JS engines in each different browser. JQuery? Prototype? Scriptaculous?
And then, once everything is selected, it all has to be configured to work together without (too many) security holes. But, of course, how much does the average developer really know about configuring a secure Linux environment with Apache? Or setting up a secure IIS? And even if the developer does know a lot about configuring all of this, wouldn't it be more productive to have him or her focused on developing actual application features rather mucking around in Apache2.conf or php.ini, or trying to figure out why their package manager can't find the right package for some random server component? How do I configure CPAN, again? Do I really need the Multiverse, or will the Universe do? Then, of course, you'll probably want an ORM, and you'll need to decide on how you want to glue all the bits and pieces together.
Not to mention keeping all of that up to date and working as new releases get rolled out… oh, and what about scaling up to meet the increased demand if you start to get really popular and get bought by Conde Nast?
Great points. Couldn't agree more. Anybody guess where I'm going with this?
Tired of worrying about infrastructure? You want to start coding now? Great, take a look at Elastic Beanstalk, Heroku, or Force.com VMForce (yeah, I know, "coming soon"). No infrastructure setup required. You still have to choose a language and a platform, I guess, but that seems unavoidable. You have to make some choices in life. However, you don't have to care about which OS or web server to use, and you don't have to manage updates of server software. AWS might all be running in VMWare within a virtualized Windows 98 stack based on a billion hand-built Commodore 64s for all I care. As long as it works. And the DBMS is a service too… you don't have to set it up, you just pick whichever one you want. When VMForce is launched, you'll have database.com as a DBMS. With Elastic Beanstalk, you have RDS or SimpleDB. With Heroku, you have PostgreSQL out of the box, with a ton of other choices available, but you don't set them up yourself, you just add them to your account, and they get set up for you.
What about security? Does your data center have 24-hour manned security, including foot patrols and perimeter inspections? Well, Salesforce does. Is your server certified by PCI, ISO, SAS70, and HIPAA? Well, AWS is, and Heroku is hosted on AWS, and they have their own operations team that monitors the system 24/7. Even Multi-Factor Authentication is just another service at AWS. And if somebody finds a security flaw in any of these platforms, it's not your problem. Somebody else can figure it out and fix it, hopefully before you even know about it. Of course, it's still important to write secure code, sanitize user inputs, parameterize SQL queries, etc., but at least that's all in _your_ code. You can focus on writing good code, and not on whether or not you accidentally configured an Apache mod incorrectly, or accidentally allowed anonymous FTP access to your web server, or if your version of PHP has a buffer overrun bug that will allow some random hacker to drop your User table.
You'll probably still need to glue some things together, and if you're doing web development, you'll still want a client-side framework so you don't have to worry too much about all the various inconsistencies between browsers, but with the infrastructure headaches out of the picture, it's easier to just start coding.
Thought I'd put together an easy to reference list of the various things that can be configured by an enterprise Mobile Device Management administrator for iOS:
Password
- Required
- No Repeating/Ascending/Descending Characters
- Require Alphanumeric
- Minimum Password length
- Minimum number of non-alphanumeric characters required
- Maximum password age (1-730 days)
- Auto-lock (1-5 minutes)
- Password History (1-50 Passwords)
- Grace Period for Device Lock (amount of time the device can be locked without prompting for a password on unlock)
- Maximum number of failed attempts (before all data on device will be erased)
Restrictions
- Allow installing apps
- Allow use of camera
- Allow Screen Capture
- Allow Automatic Sync while Roaming
- Allow voice dialing
- Allow In App Purchase
- Allow Multiplayer Gaming
- Allow Adding Game Center Friends
- Force Encrypted Backups
- Applications
- Allow use of YouTube
- Allow use of iTunes Music Store
- Allow use of Safari
- Enable autofill
- Force fraud warning
- Enable JavaScript
- Block Pop-ups
- Accept Cookes: Always, Never, From Visited Sites
- Allow Explicit Music & Podcasts
- Allowed Content Ratings
- Movies: Don't Allow Movies, G, PG-13, R, NC-17, Allow All Movies
- TV: Don't Allow TV Shows, TV-Y, TV-Y7, TV-G, TV-PG, TV-14, TV-MA, Allow All TV Shows
- Apps: Don't Allow Apps, 4+, 9+, 12+, 17+, Allow All Apps
Wi-Fi
- Service Set Identifier (SSID)
- Hidden Network (if target network is set to not broadcast)
- Security Type: Any (Personal), None, WEP, WPA/WPA2, WEP Enterprise, WPA/WPA2 Enterprise, Any (Enterprise)
- Password
VPN
- Connection Name
- Connection Type: L2TP, PPTP, IPSec (Cisco), Cisco AnyConnect, Juniper SSL, F5 SSL, Custom SSL
- Server Hostname or IP Address
- Account
- User Authentication: Password, RSA SecurID
- Shared Secret
- Send All Traffic (Route all network traffic through VPN)
- Proxy Setup
Email
- Account Description
- Account Type: IMAP, POP
- User Display Name
- Email Address
- Mail Server and Port
- Authentication Type: None, Password, MD5 Challenge-Response, NTLM, HTTP MD5 Digest
- Password
- Use SSL
Exchange ActiveSync
- Account Name
- Exchange ActiveSync Host (Exchange Server)
- Use SSL
- Domain
- User
- Email Address
- Password
- Past Days of Mail to Sync: No Limit, 1 Day, 3 Days, 1 Week, 2 Weeks, 1 Month
- Authentication Credential Name
- Authentication Credential
- Include Authentication Credential Passphrase
LDAP
- Display Name
- Account Username
- Account Password
- Account Hostname
- Use SSL
- Search Settings
CalDAV
- Account Description
- Account Hostname and Port
- Principal URL
- Account Username
- Account Password
- Use SSL
CardDAV
- Account Description
- Account Hostname and Port
- Principal URL
- Account Username
- Account Password
- Use SSL
Subscribed Calendar
- Description
- URL
- Username
- Password
- Use SSL
Web Clips (web pages saved to the home screen as bookmarks)
- Label
- URL
- Removable (yes/no)
- Icon
- Precomposed Icon
- Full Screen
Credentials
- Specify PKCS1 and PKCS12 certificates needed to authenticate access to your network
SCEP
- URL for SCEP server
- Name
- Subject (representation of X.500 name)
- Subject Alternative Name Type (None, RFC 822 Name, DNS Name, Uniform Resource Identifier)
- Subject Alternative Name Value
- NT Principal Name
- Challenge
- Key Size: 1024, 2048
- Use as digital signature
- Use for key encipherment
- Fingerprint (hex string)
MDM
- MDM Server URL
- Check in URL
- Topic (Push notification topic for management messages)
- Identity: Add credentials in Credentials payload, SCEP
- Sign Messages: yes/no
- Access Rights granted to remote administrators:
- Query Device for:
- Device Information
-
- unique device identifier (UDID)
- device name
- iOS version
- device model name and hardware version
- serial number
- overall and available storage capacity
- IMEI number
- the modem firmware version
- SIM card ICCID
- and MAC addresses for integrated Wi-Fi and Bluetooth
- carrier currently being used
- the carrier specified by the current installed SIM card
- the version of the carrier settings (APN) data
- assigned phone number
- whether or not data roaming is currently allowed
- list of configuration profiles installed
- list installed security certificates and expiry dates
- list of enforced restrictions
- hardware encryption capability
- whether an unlock passcode is set
- installed applications (with App identifier, name, version, and size)
- a list of any application provisioning profiles with expiration dates.
- General Settings
- Security Settings
- Network Settings
- Restrictions
- Configuration Profiles
- Applications
- Provisioning Profiles
- Add / Remove:
- Configuration Profiles
- Provisioning Profiles
- Security
- Change device password
- Remote Wipe
- Apple Push Notification Service
- Use Development APNS Server
Advanced
- Access Point Name (APN): The name of the GPRS access point
- Access Point User Name
- Access Point Password
- Proxy Server and Port
There seems to be a lot of industry buzz lately around the concept of "gamification", and the idea is basically one of applying game mechanics to the world of business to motivate employees or customers. Bunchball has done a really nice job with their Gamification 101 white paper of illustrating how gamification can work in a variety of circumstances, and why you should be using it in your business. It's a good read, and a good place to get started learning the concepts. Some examples of gamification that they give are frequent flyer programs, where customers earn points and "level up" to different statuses over time, and Starbucks' use of the Foursquare to check in and win "trophies or badges". Another good resource is the Gamification Encyclopedia at Gamification.org.
Mechanics, Dynamics, and Aesthetics (MDA)
So what makes a game? Bunchball discuss the terms "game mechanics" and "game dynamics" in their white paper, and those terms come from a game design approach called MDA (Mechanics, Dynamics, Aesthetics), described by Hunicke, LeBlanc, and Zubek in their article MDA: A Formal Approach to Game Design and Game Research. The idea is basically that a game designer creates various rules for a game (Mechanics). These rules then work together (in sometimes unexpected ways) to create a system (Dynamics). And a player experiences these Dynamics through the Aesthetics of the game, which they categorize into things like "Challenge", "Discovery", or "Narrative". So, the game designer sets up the game by manipulating the Mechanics, and the player experiences the game through the Aesthetics. Put simply, if the designer can directly manipulate something by changing the rules of the game, it is part of the game mechanics. Dynamics are manipulated indirectly by the designer, and aesthetics are experienced by the player. Game design is complicated, then, because the experience of the player is two steps removed from the rules set forth by the designer.

A simple example can be illustrated with the game of Poker. The mechanics of the game involve dealing cards, anteing, and betting. The dynamics of the game have emerged over time to include things like bluffing. And the aesthetics of the game include things like fellowship (it's a good game to play with friends) and challenge (your opponents present many obstacles to winning).
Success!
So what makes a game fun? Every game uses game mechanics, but many have been utter failures. What makes a gamification strategy successful? It's easy to throw together leader boards, loyalty programs, and point systems, but how do you actually drive behavior with gamification? And what exactly is a game anyway? It's one of those things that you know when you see it, but how do you actually define it? Chris Crawford offers an interesting definition of "game" in his book, Chris Crawford on Game Design, Basically, if there is no competition (either amongst players or against some form of AI), then what you have is a puzzle, not a game. Additionally, if you have no influence over how your opponent is performing, then that competition isn't a game either. By this definition, solitaire is a puzzle, because there is no competition. A drag race is a competition, but not a game, because you can't slow the other car down in any way. However, a race where you are allowed to run your opponent off the road is a game.
So, does a successful gamification strategy need to follow this definition? Does there need to be competition, and should employees or customers be able to alter other's ability to perform? Perhaps not, but competition is likely to be important in any successful gamification strategy.
Pitfalls
So, what are some pitfalls of game design? If some players are able to get too far ahead of the pack, does it create a disincentive for the rest of the players? How can you reward top players without discouraging everyone else? Consider Monopoly: The game starts out fun for everyone, but as one or more players start buying up all the property, the "poorer" players get less and less interested in completing a game that they have very little chance of winning. How could the mechanics of Monopoly be adjusted to keep everyone engaged? It's important to consider positive and negative feedback loops in the game. Monopoly has a strong positive feedback loop. The more property a player has, the more money they make from other players, which they use to purchase more and more property. To cancel this out, one could adjust the game mechanics to include, say, theft. This could introduce a negative feedback loop by making players who are doing well more likely to have property stolen by players on the poverty side of the equation.
Frequent flyer programs have a similar problem. People who fly frequently form loyalties to airlines because they have so many points built up that they are able to reap the benefits of the program. People with few points have little incentive to be loyal to any specific airline because they are a long way from "leveling up" and seeing any tangible benefit from the program. The airlines probably don't care as much about these infrequent flyers, but they may be missing out on nurturing loyalties in people who may become more frequent flyers in the future. These "players" could be incentivized to by being entered in a drawing each time they fly, or randomly getting free drinks or being upgraded to first class when seats are available.
Overall
Overall, I think it's really exciting that business leaders are starting to consider employee and customer motivation from the perspective of the game designer, and it's nice to see some formalized thought being put forward that takes some lessons from "regular" game designers and researchers. It will be interesting to see what innovative groups like Bunchball come up with over time.