From the desert to the sea: Google Voice Search experiments

(Cross posted from the Google Australia Blog)

As an engineer I like to solve problems and I like to test stuff, the bigger the better. A couple of months ago I was intrigued by a challenge: Australians have the second highest smartphone penetration in the world (second only to Singapore) but we have one of the lowest usage rates of Google Voice Search. This had some of us scratching our heads, because Google Voice Search understands our Australian accents.

So we thought it would be fun to show Australians how well Google Voice Search works by conducting a series of experiments in remote Australian locations. I went to the middle of the South Australian desert with James, a fellow engineer. We found some of the hottest, reddest earth in the country to test a long distance voice search - 50 metres away from the phone! Here’s our adventure:



Meanwhile, our fellow Google engineers Noel and Alice went up north to the beautiful Great Barrier Reef, to test Google Voice Search underwater.



Now that you’ve seen what Google Voice Search can do, we hope these experiments will inspire you to try it yourself in a more everyday setting, and with your broadest Aussie accent!

You can use Google Voice Search on Android phones by pressing the microphone button on the homescreen or downloading the app. You can also download the app for the iPhone here.

Posted by Mike Lawther, Software Engineer.

Introducing Android WebDriver

[This post is by Dounia Berrada, an engineer on the EngTools team. — Tim Bray]

Selenium WebDriver is a browser automation tool which provides a lightweight and elegant way for testing web apps. Selenium WebDriver is now available as an SDK extra in the Android SDK, and supports 2.3 (Gingerbread) and onwards!

Whether or not your site is optimized for mobile browsers, you can be sure that users will be accessing it from their phones and tablets. WebDriver makes it easy to write automated tests that ensure your site works correctly when viewed from the Android browser. We’ll walk you through some basics about WebDriver and look at it in action.

WebDriver Basics

WebDriver tests are end-to-end tests that exercise the web application just like a real user would. WebDriver models user interactions with a web page such as finger flicks, finger scrolls and long presses. It can rotate the display and interact with HTML5 features such as local storage, session storage and the application cache. Those tests run as part of an Android tests project and are based on Junit. They can be launched from Eclipse or the command line. WebDriver tests can be wired with a continuous integration system and can run on phone and tablet emulators or real devices. Once the test starts, WebDriver opens a WebView configured like the Android browser and runs the tests against it.

WebDriver is an Android SDK extra and can be installed following these instructions. Once you’ve done that you’ll be ready to write tests! There is a comprehensive WebDriver user guide on the Selenium site, but let’s start with a basic example using www.google.com to give you a taste of what’s possible.

Getting Started

First, create an Android project containing an empty activity with no layout.

public class SimpleAppActivity extends Activity {
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
}
}

Then create the Android test project that will contain the tests. WebDriver will create the WebView and set the layout automatically in the main Activity.

Let’s write a test that opens the Google home page on Android and issues a query for “weather in San Francisco”. The test will verify that Google returns search results, and that the first result returned is giving the weather in San Francisco.

public class SimpleGoogleTest extends ActivityInstrumentationTestCase2<SimpleAppActivity> {

public void testGoogleShouldWork() {
// Create a WebDriver instance with the activity in which we want the test to run
WebDriver driver = new AndroidDriver(getActivity());
// Let’s open a web page
driver.get("http://www.google.com");

// Lookup for the search box by its name
WebElement searchBox = driver.findElement(By.name("q"));

// Enter a search query and submit
searchBox.sendKeys("weather in san francisco");
searchBox.submit();

// Making sure that Google shows 11 results
WebElement resultSection = driver.findElement(By.id("ires"));
List<WebElement> searchResults = resultSection.findElements(By.tagName("li"));
assertEquals(11, searchResults.size());

// Let’s ensure that the first result shown is the weather widget
WebElement weatherWidget = searchResults.get(0);
assertTrue(weatherWidget.getText().contains("Weather for San Francisco, CA"));
}
}

Now let’s see our test in action! WebDriver will create a WebView with the same configuration as the Android browser in the main UI thread, i.e. the activity thread. The activity will display the WebView on the screen, allowing you to see your web application as the test code is executing.

Interaction Testing

We’ve mentioned that WebDriver supports creating advanced gestures to interact with the device. Let’s use WebDriver to throw an image across the screen by flicking horizontally, and ensure that the next image in the gallery is displayed.

WebElement toFlick = driver.findElement(By.id("image"));
// 400 pixels left at normal speed
Action flick = getBuilder(driver).flick(toFlick, 0, -400, FlickAction.SPEED_NORMAL)
.build();
flick.perform();
WebElement secondImage = driver.findElement(“secondImage”);
assertTrue(secondImage.isDisplayed());

Now, let’s rotate the screen and ensure that the image displayed on screen is resized.

assertEquals(landscapeSize, secondImage.getSize())
((Rotatable) driver).rotate(ScreenOrientation.PORTRAIT);
assertEquals(portraitSize, secondImage.getSize());

What if your test reveals a bug? You can easily take a screenshot for help in future debugging:

File tempFile = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);

Find Out More

If this has whetted your appetite and you’d like to know more, go ahead and install the Android WebDriver, take a look at the documentation on the Selenium project’s wiki, or just browse the javadocs. For questions and feedback not only of the Android WebDriver but also its desktop brethren, please join webdriver@googlegroups.com. For announcements keep an eye on http://seleniumhq.wordpress.com/.

Changes to Library Projects in Android SDK Tools, r14

Last week, we released the SDK for Android 4.0 and a new set of developer tools, now in revision 14. The updated tools include a lot of build changes, many that improve build performance. Also included is an under-the-hood change in how libraries are used by main projects — a first step in improving library support and code reusability. While the change should have little impact on existing projects, some developers have had issues when migrating to the updated tools. Please read below for more information about the change to library projects and how to solve migration issues.

Previously, library projects were handled as extra resource and source code folders to be used when compiling the resources and the application’s source respectively. While this worked fine for most cases, there were two issues.

1. Developers asked us for the ability to distribute a library as a single jar file that included both compiled code and resources. The nature of Android resources, with their compiled IDs prevented this.

2. The implementation of the library projects was extremely fragile in Eclipse. Adding extra source folders outside of the project folders is non-trivial when it needs to be handled automatically, in a way that doesn’t expose a user’s local installation path (which is required for people working in teams through a source control system such as SVN or git).

For r14, we decided to fix both issues at once, by moving to a compiled-code based library mechanism. This solves the implementation fragility in Eclipse and will allow us to, later, enable distribution of libraries as a single jar file.

As you might have seen in the release notes, moving to this new mechanism can affect existing projects in some cases, but there are simple fixes.

The first impact of this change is that the new library project requires the resource IDs generated by libraries to be non final. This prevents the Java compiler from inlining the values in the library code, and therefore prevents usage of the switch statement in the library code. To address such occurrences in your code, Eclipse provides a refactoring action to convert from switch statements to if/else (see here).

Second, some projects may not have been properly migrated to the new mechanism, resulting in projects that fail to compile, with errors such as duplicated classes being added in the dex build step. ADT 14 should have migrated older projects to the new mechanism but the fragility of the old mechanism may have prevented it from happening. This makes projects reference the libraries twice, using both the old and new mechanisms, which then triggers the libraries classes being packaged twice. If you see this in your projects, look in the Package Explorer for extraneous source folders named with the pattern <libraryname>_src. The screenshot to the right shows an example of this.

To fix the project, you must remove the extraneous source folders with the following steps:

  • Right click source folder and choose Build Path > Remove from Build path
  • A dialog will pop up. In it, make sure to check “Also unlink the folder from the project” to completely remove the folder.

With this change to library projects, we pave the way to better support for reusable components. We will continue working to make components easier to create, work with, and manage. Our goal is to make it easy for developers to create apps with great user experiences that easily adapt to all form factors.

Some developers have also told us that they only use library projects internally, that they don’t need to distribute binary versions and would prefer to continue with a source-based mechanism. We are investigating how we could support this alongside the new mechanism.

Finally, I wanted to point out that we are tracking a few known issues (and workaround for them) in the current r14 tools at this page: http://tools.android.com/knownissues.

We are working on a tools update that will include fixes for most of these. We are hoping to have it out shortly.

Google Catalogs presents the coveted Neiman Marcus Christmas Book and debuts the Fantasy Gifts Retrospective--for free on your tablet

(Cross-posted from the Google Commerce blog)

Neiman Marcus is known for the outrageously beautiful and unique merchandise showcased in their annual Christmas Book. Think of the most amazing gifts you can imagine -- a beautiful Chanel handbag, a Orefici watch, or perhaps your very own tailored mermaid suit or customizable cupcake car.

For this holiday season, Google Catalogs has collaborated with Neiman Marcus to offer you a free, browsable copy of this year’s Christmas Book. We’ve also created the very first historic compilation of Neiman Marcus Fantasy Gifts from the past ten years. It’s called the Fantasy Gifts Retrospective and you’ll only find it on the Google Catalogs app for iPad, available for free in the App Store.

To celebrate the Fantasy Gifts Retrospective, we’ve picked out our favorite gifts from the past 10 years! Sadly, gifts from prior years are no longer for sale, but we trust you'll enjoy the 2011 gifts and the walk down memory lane.




New Public APIs in ICS

Since Android is open-source, anyone can look at the code and see how it works inside. If you do this, you’ll notice that most but not all of the APIs are publicly documented.

If they’re publicly documented, they’re part of what we consider the Android Application Framework. This means their tests appear in the Compatibility Test Suite (CTS) so that our hardware partners have to prove that the APIs work, and that we promise to try very hard not to change them and thus break your code.

In almost every case, there’s only one reason for leaving APIs undocumented: We’re not sure that what we have now is the best solution, and we think we might have to improve it, and we’re not prepared to make those commitments to testing and preservation.

We’re not claiming that they’re “Private” or “Secret” — How could they be, when anyone in the world can discover them? We’re also not claiming they’re forbidden: If you use them, your code will compile and probably run. And in fact we know of quite a few apps out there whose developers have used undocumented APIs, often to good effect. It’s hard to get too upset about this in cases where there’s a useful API that we haven’t gotten around to stabilizing.

But the developers who use those APIs have to be prepared to deal with the situation that arises when we move them from the undocumented outside into the Android Application Framework. Fortunately, this is reasonably straightforward. Also we take a close look at Android Market, using our in-house analytics tools, to get a feel for the impact when we know one of these changes is coming.

There are a few such changes coming up in the Android 4.0 “Ice Cream Sandwich” (ICS) release of Android. We wanted to take the opportunity to combine these words on undocumented APIs with some specifics about the changes.

Calendars

Let’s start with the good news: As of ICS, the Android Framework will include a fully-worked-out set of APIs for accessing Calendar data. You can guess the bad news: Quite a few developers have built apps (including many good ones) using the undocumented Calendar APIs, some using fairly low-level access to the calendar database. Unfortunately, these integrations were unsupported, and prone to breakage by platform updates or OEM customization of calendar features.

We want to see lots of good calendar apps and extensions that work reliably across Android devices, and aren't broken by platform updates. So we decided to create a clean API, including a comprehensive set of Intents, to manage calendar data in ICS. Now anyone can code against these new APIs and know that Android is committed to supporting them, and that partners have to support these APIs as part of CTS.

Once the new APIs arrive, you’re going to have to update your apps before they’ll run correctly on ICS while still working on older releases. There are a variety of techniques for doing that, many of which have been featured on this blog, including reflection and lazy loading. Recently, we introduced Multiple-APK support, which could also be used to help with this sort of transition.

Text To Speech

Android has never really had a text-to-speech API at the Framework level, but there was unofficial access at the C++ level. With ICS, we will have a fully-thought-through application-level API running on Dalvik, so you can access it with ordinary Java-language application code.

The old C++ API will no longer be supported, but we’ll have a compatibility layer that you can use to bridge from it to the new API. We think it should be easy to update for ICS with very little work.

Doing the Right Thing

We recognize that this means some work for developers affected by these changes, but we’re confident that Android programs in general, and both Calendar and TTS apps in particular, will come out ahead. And we also think that most developers know that when they use undocumented APIs, they’re making a commitment to doing the right thing when those APIs change.

Android 4.0 Platform and Updated SDK Tools

ICS logo

Today we are announcing Android 4.0, Ice Cream Sandwich — a new version of the platform that brings a refined, unified user experience for phones, tablets, and more.

Android 4.0 builds on the things people love most about Android — efficient multitasking, rich notifications, customizable home screens, resizable widgets, and deep interactivity — and adds powerful new ways of communicating and sharing. It includes many great features for users, including social and sharing integration, network data usage control, innovative connectivity and camera options, and an updated set of standard apps.

For developers, Android 4.0 introduces many new capabilities and APIs. Here are some highlights:



Unified UI toolkit: A single set of UI components, styles, and capabilities for phones, tablets, and other devices.

Rich communication and sharing: New social and calendar APIs, Android Beam for NFC-based instant sharing, Wi-Fi Direct support, Bluetooth Health Device Profile support.

Deep interactivity and customization: Improved notifications, lockscreen with camera and music controls, and improved app management in the launcher.

New graphics, camera, and media capabilities: Image and video effects, precise camera metering and face detection, new media codecs and containers.

Interface and input: Hardware-accelerated 2D drawing, new grid-based layout, improved soft keyboard, spell-checker API, stylus input support, and better mouse support.

Improved accessibility: New accessibility APIs and text-to-speech APIs for writing new engines.

Enhancements for enterprise: Keychain and VPN APIs for managing credentials and connections, a new administrator policy for disabling the camera.

For a complete overview of what’s new for users and developers, please read the Android 4.0 Platform Highlights.

Alongside the new Android platform, we are releasing new versions of the SDK Tools (r14) and ADT Plugin (14.0) for Eclipse. Among the highlights are:

  • Improved build performance in Ant and Eclipse

  • Improved layout and XML editors

To get started developing on Android 4.0, visit the Android Developers site for information about the Android 4.0 platform, the SDK Tools, and the ADT Plugin.

If you have already developed and published apps, we encourage you to download the Android 4.0 platform now, to begin testing your app before devices arrive in stores.



Check out the video below for a closer look at Android 4.0 in action.



Unwrapping Ice Cream Sandwich on the Galaxy Nexus

(Cross-posted on the Official Google blog)

Beaming a video with a single tap or unlocking a device with only a smile sounds like science fiction. Now, you can actually do these things (and more) with a phone that fits in the palm of your hand.

Wednesday morning in Hong Kong—together with Samsung—we unveiled Galaxy Nexus, the first phone designed for the latest release of Android 4.0, also known as Ice Cream Sandwich.



With a super slim profile, Galaxy Nexus features a 4.65” Contour Display with true high definition (720p) resolution and a lightning-fast dual core 1.2ghz processor combined with 4G LTE or HSPA+ technology. Galaxy Nexus also features the latest in software: Ice Cream Sandwich makes Android simple and beautiful, and takes the smartphone to beyond smart.

Beauty and simplicity
With Ice Cream Sandwich, our mission was to build a mobile OS that works on both phones and tablets, and to make the power of Android enticing and intuitive. We created a new font that’s optimized for HD displays and eliminated all hardware buttons in favor of adaptable software buttons. We also dramatically improved the keyboard, made notifications more interactive and created resizable widgets.

The desktop-class browser is significantly faster, featuring a refined tab manager and the ability to sync your bookmarks with Google Chrome. Ice Cream Sandwich also features the best mobile Gmail experience to date, with a new design that lets you quickly swipe through your inbox and search messages even when you’re offline. Calendar boasts a clean new look and you can zoom into your schedule with a pinch.

Connect and share
People are at the heart of Ice Cream Sandwich. We rethought how you browse your contacts with the new People app, which combines high-resolution photos and updates from Google+ and other social services. It’s also easier to capture and share your life with family and friends. Galaxy Nexus sports a high-end camera with zero shutter lag, automatic focus, top notch low-light performance and a simple way to capture panoramic pictures. Shoot amazing photos or 1080p video, and then edit and share them directly from your phone.

Beyond smart
Galaxy Nexus isn’t just a smartphone—it’s beyond smart. Ice Cream Sandwich gives you complete control over the amount of mobile data you use by helping you better understand and manage it. We’re also introducing Android Beam, which uses near field communication (NFC) to instantly share webpages, YouTube videos, maps, directions and apps by simply tapping two phones together. Face Unlock uses state-of-the-art facial recognition technology to unlock your phone with nothing more than a smile.

This weekend marks the third birthday of the G1, the first-ever Android phone. Nine releases later, more than 550,000 Android devices are activated daily.

Starting in November, Galaxy Nexus will be available in the United States, Canada, Europe and Asia. Check out the Nexus website for a product tour and more info.

More Google Wallet merchants are live. Now you can pay AND save in a single tap.

(Cross-posted on the Official Google Blog)

We’re hearing from people at check-out counters throughout the country that paying with your phone is a little like magic. Just look at the ecstatic reaction on the faces of our friends who made their first Google Wallet purchases last Thursday.



Today, our partners American Eagle Outfitters, The Container Store, Foot Locker, Guess, Jamba Juice, Macy’s, OfficeMax and Toys“R”Us are rolling out an even better Google Wallet experience. For the first time ever in the U.S., at these select stores, you can not only pay but also redeem coupons and/or earn rewards points—all with a single tap of your phone. This is what we call the Google Wallet SingleTap experience.

With Google Wallet in hand, you can walk into a Jamba Juice, American Eagle Outfitters or any other partner store. Once you’ve ordered that Razzmatazz smoothie or found the right color Slim Jean, head straight to the cashier and tap your phone to pay and save—that’s it. You don’t have to shuffle around to find the right coupon to scan or rewards card to stamp because it all happens in the blink of an eye.

The Offers tab in Google Wallet has been updated to include a new "Featured Offers" section with discounts that are exclusive to Google Wallet. Today, these include 15% off at American Eagle Outfitters, 10% off at The Container Store, 15% off at Macy’s and an all-fruit smoothie for $2 at Jamba Juice. There are many more Google Wallet exclusive discounts to come, and you can save your favorites in Google Wallet so they’ll be automatically applied to your bill when you check out.




Organizing loyalty cards in your wallet is getting easier too. Today, Foot Locker, Guess, OfficeMax and American Eagle Outfitters are providing loyalty cards for Google Wallet so you can rack up reward points automatically as you shop. More of these are on the way.

One more thing—in response to user feedback, we’ve improved transaction details for the Google Prepaid Card with real-time transaction information including merchant name, location, dollar value and time of each transaction. Here’s what it looks like:






Finally, a special thanks to Chevron, D’Agostino, Faber News Now, Gristedes Supermarkets and Pinkberry who are now also working to equip their stores to accept Google Wallet.

It’s still early days for Google Wallet, but this is an important step in expanding the ecosystem of participating merchants to make shopping faster and easier in more places. If you’re a merchant and want to work with us to make shopping easier for your customers and connect with them in new ways, please sign up on the Google Wallet site. And if you’re a shopper and want to purchase a Nexus S 4G phone from Sprint with Google Wallet, visit this page.

Start the conversation with Google Translate for Android


Mobile technology and the web have made it easier for people around the world to access information and communicate with each other. But there’s still a daunting obstacle: the language barrier. We’re trying to knock down that barrier so everyone can communicate and connect more easily.

Earlier this year, we launched an update to Google Translate for Android with an experimental feature called Conversation Mode, which enables you to you translate speech back and forth between languages. We began with just English and Spanish, but today we’re expanding to 14 languages, adding Brazilian Portuguese, Czech, Dutch, French, German, Italian, Japanese, Korean, Mandarin Chinese, Polish, Russian and Turkish.




To use Conversation Mode, speak into your phone’s microphone, and the Translate app will translate what you’ve said and read the translation out loud. The person you’re speaking with can then reply in their language, and Conversation Mode will translate what they said and read it back to you.

This technology is still in alpha, so factors like background noise and regional accents may affect accuracy. But since it depends on examples to learn, the quality will improve as people use it more. We wanted to get this early version out to help start the conversation no matter where you are in the world.

We’ve also added some other features to make it easier to speak and read as you translate. For example, if you wanted to say “Where is the train?” but Google Translate recognizes your speech as “Where is the rain?”, you can now correct the text before you translate it. You can also add unrecognized words to your personal dictionary.

When viewing written translation results, you can tap the magnifying glass icon to view the translated text in full screen mode so you can easily show it to someone nearby, or just pinch to zoom in for a close-up view.



Tap the magnifying glass icon to view translations full screen.


Finally, we’ve also optimized the app for larger screens like your Android tablet.

While we work to expand full Conversation Mode to even more languages, Google Translate for Android still supports text translation among 63 languages, voice input in 17 of those languages, and text-to-speech in 24 of them.

Download the Google Translate app in Android Market — it’s available for tablets and mobile phones running Android 2.2 and up.


An enhanced Google Docs experience on Android tablets

(Cross-posted from the Google Docs blog)

Earlier this year, we introduced the Google Docs app for Android. Since then, many users have downloaded the app and enjoyed the benefits of being able to access, edit and share docs on the go.

Today’s update to the app makes Google Docs work better than ever on your tablet. With an entirely new design, we’ve customized the look to make the most of the larger screen space on tablets. The layout includes a three-panel view, which allows you to navigate through filters and collections, view your document list, and see document details, all at once.

Looking at the details panel on the right side, you can see a thumbnail to preview a document and its details before opening it. From the panel, you can see who can view or edit any doc.

New 3-panel view for improved browsing



Autocomplete makes sharing with others on the go even easier


These features are now available in 46 languages on tablet devices with Android 3.0+ (Honeycomb) and above.

You can download the app from the Android Market and let us know what you think in the comments or by posting on the forum. Learn more by visiting the help center.

Android Market Featured-Image Guidelines

[This post is by Natascha Bock, a Product Marketing Manager on Android. — Tim Bray]

With the latest Android Market update, our editorial team can use your 1024 x 500 “Featured Image” to promote your app on tablets, phones, and the Web. The image can be used on the home page on all versions of Android Market (Web, tablet and phone), on your product page in the Web and tablet versions, and on current and future top-level Android Market pages on phones.

Creating a Featured Image that will do a great job of highlighting your app requires special design consideration.

Not Really Optional

While many promotional assets are listed as “optional” for the publishing site, we strongly recommend treating them as required. To start with, a Featured Image is required if your app is to be featured anywhere within Android Market. They’re a good idea anyhow; they enhance your product page, making your game or app more attractive to end-users (and more likely to be considered for featuring by our editorial team).

There’s nothing optional about the size, either; it has to be 1024 x 500 pixels.

Do’s and Dont’s

Your graphic is not an ad, it’s a teaser. It’s a place for bold, creative promotional images.

Vivid background colors work best. Black and white are problems because those are the backgrounds used by the mobile-device and Web versions of Android Market.

Limit Text to your app name and maybe a few additional descriptive words. Anything else will be unreadable on phones, anyhow.

Do: Make the graphic fun
and enticing.
Don't: Create a text-heavy
advertising-style graphic.
Do: Use colors that stand out on
black or white backgrounds.
Don't: Let the graphic fade into
the background.
Do: Promote your brand prominently.
Don't: Overload the graphic with details.

Scaling

Your image has to be designed to scale; it will need to look good both in a full-size Web browser and on a little handset. You can rely on the aspect ratio being constant, but not the size. Here’s a tip: Try resizing your image down to 1 inch in width. If it still looks good and conveys your brand message, you have a winner.

On the Web:


On a tablet:



On a big phone:



On a small phone:

More Dont’s

  • Device imagery is tempting, but becomes dated fast, and may be inappropriate if your user’s device looks entirely different.

  • In-app screenshots are inappropriate because your product page already includes a place for these.

  • Just using your app icon is a failure of imagination. You have more room; put it to good use!

Consider the Context

Given the size of the form factor, the phone is the most challenging channel for your image. Below we have both the “good” and “bad” sample images in that context:

Don’t Forget

A 1024 x 500 Featured Image is required for feature placement consideration. Don't miss out on the opportunity!