One of the problems I mentioned in my recent post about Java ME was the lack of reliable emulators on which Java ME applications can be tested. As a result of that, during the years the need to have every Java ME application tested on every mobile telephone model has become a MUST. Given the huge growing number of mobile telephones models, the differences that sometime exist when comparing the same model operating in different countries and\or with different carriers that need has become nearly an impossible one. The huge number of mobile telephones models, their differentiation and even the differences we can find when comparing the same model working with different carriers… all that creates a serious problem for Java ME developers who want their application to be available for as many people as possible world wide.
I have recently found DeviceAnywhere, a company that provides an over the web solution aiming to assist Java ME developers world wide with testing their Java ME application on various mobile telephones located in different countries. In this post I am trying to evaluate their solution and answer the question whether it solves the problem mentioned above (lack of reliable emulators). DeviceAnywhere solution provides an easy to use web based mechanism through which every Java ME can be tested on the real handsets allowing to test on various models located in different countries and connected with various wireless carriers.
I have contacted with DeviceAnywhere and following their approval and a trial account they arranged I tried to use their over the web solution to test Jacado Klondike Solitaire on Sony Ericsson C902 (O2 UK), Sony Ericsson W910i (T-Mobile UK) and on Sony Ericsson C702 (Vodafone UK).
Based on the information DeviceAnywhere presets on their web site it seems they have every mobile telephone model (on which a Java ME application should be tested) physically wired up to their system.
I started the process of testing a Jacado Klondike Solitaire JAR file on the selected models. The web interface they company provides is easy to use and their online demo is very useful as a jump start guidelines tutorial.
A Java Web Start Java application can be executed on their web site. Using that application the testing is performed.
I started by trying to acquire Sony Ericsson W910i. The whole system works very slow. Given the fact that I am using a PC with 2MB RAM connected to the web with a 8MB ADSL connection it seems the problem is somewhere on the server side.
After spending some time… too much IMHO… I gave up trying to cause the remote physical handset to ‘goto’ a specific URL address on which I hosted my Java ME application’s JAR file. After talking with DeviceAnywhere I tend to believe the reason for this malfunction is probably some sort of networking problem on my end.
Based on my experience so far I can conclude that assuming this solution does work it cannot be a real replacement (in all cases) for testing a Java ME application on the real handset. Based on the limitations I found during my trial period, this solution can fit specific cases only.
Slow Response
The keys slow response limits this solution for simple standard applications. DeviceAnywhere solution cannot replace the real handset when testing high quality games and action games in particular.
Unreliable Graphics Quality
The DeviceAnywhere video image isn’t an analogue video capture of the device screen. The DeviceAnywhere video image is digitized directly from the video chip of the handset. I would have expected to have a real video capture using a separated digital camera. Using DeviceAnywhere solution requires us to assume the handset’s LCD is as accurate as the PC we use.
Pure Sound Quality
The sound quality over the web is not good enough to test high quality sounds as well as stereo effects and similar.
Missing Keyboard Interaction
DeviceAnywhere doesn’t allow us to interact with the mobile telephones’ keys via an hardware robotic mechanism. That requires us to assume thet DeviceAnywhere solution interacts digitally with the device in an accurate way.
Nevertheless, using DeviceAnywhere surely might be sufficient in those specific cases in which there is a need to overcome a bug identified in a specific handset working with a specific wireless carrier. Based on my experience during the years those cases do happen (e.g. a Java ME specific version for a specific mobile telephone model works fine and passes all QA tests… later, having that same very version running on the same telephone model only working with another carrier the sound doesn’t work… probably due to some bug resulted from specific modifications the mobile telephone manufacturer introduced for that specific wireless carrier).
