Software QA focused on developers – and not the cloud

Remember the old saying, “Everyone is talking about the weather, but nobody is doing anything about it?” That’s pretty much the case when it comes to using the cloud as part of a software QA process.

In research conducted by SD Times in July, we asked, “At what stage is your company, or companies that you consult, using the cloud for software testing?” Very few respondents indicated that they use the cloud in this way:

At what stage is your company, or companies that you consult, using the cloud for software testing?

We are using the cloud for software testing on a routine basis 7.9%
We are experimenting with using the cloud for software testing 17.3%
We are studying the technology but have not started yet 26.7%
No plans to use the cloud for software testing 39.6%
Don’t know 8.4%

When it comes to software quality assurance, what happens in Vegas stays in Vegas most of the time. Or to put it more clearly, the activity usually but not always is conducted by the organization’s employees:

Does your company outsource any of its software quality assurance or testing?

Yes, all of it 4.4%
Yes, some of it 26.6%
No, none of it 65.0%
Don’t know 3.9%

There’s no one favorite model about where testing lives. Is it part of the development group? Is it separate? Is it sometimes separate and sometimes integrated? The answers were surprisingly evenly split.

Does your organization have separate development and test teams?

All test and development teams are integrated 30.2%
All development teams and test/QA teams are separate 32.7%
Some development and test/QA teams are separate, some are integrated 34.6%
Don’t know 2.4%

Enterprise developers can’t simply throw the code over the metaphorical wall once it was completed and let other IT staff take complete responsibility for quality assurance – even after deployment.

We asked, “Who is responsible for internally-developed application performance testing and monitoring in your company?” with separate answers for prior to deployment and after deployment. The answers showed that developers still responsibility after deployment – and sysadmins were in the loop during the development process.

Who is responsible for internally-developed application performance testing and monitoring in your company?

Software/Application Developers prior to deployment 60.8%, after deployment 47.7%
Software/Application Development Management prior to deployment 52.8%, after deployment 53.8%
Testers prior to deployment 50.3%, after deployment 41.5%
Testing Management prior to deployment 48.7%, after deployment 38.5%
IT top management (development) (VP or above) prior to deployment 36.7%, after deployment 34.6%
Systems administrators prior to deployment 24.1%, after deployment 45.4%
Networking personnel prior to deployment 21.5%, after deployment 31.5%

When it comes to software quality assurance, one thing is for certain: We are all in it together.


Don’t roll your own math

When it comes to writing code with advanced numerical functions, my advice is clear: Use libraries. Don’t roll your own algorithms.

Generally speaking, I’m a fan of modular code reuse, especially for complex functions like UI controls, database access drivers, PDF generation or managing images. Most of the time, it’s a good idea to find open-source components that will get your job done, or license commercial reusable components. Sometimes, though, it makes more sense to write your own functionality.

Numerical libraries are the exception. Unless you are in the math business, resist the temptation to write your own Fast Fourier Transform (FFT) functions, random number generators, basic linear algebra subprogram (BLAS), wavelets, Eigenvalues, partial differential equations – you get the picture.

This came up thanks to an email from an SD Times subscriber:

I'm an IT consultant in the software arena and would like to ask you a question on buying mathematical algorithms vs. programming them yourself. Especially for complicated mathematical subroutines, is it cost-effective to subscribe to an algorithm library, or let your programmers do all the work?

Advanced numerical algorithms are very hard to get right. Simply writing the basic code is complicated – and so is the testing of that code, to make sure that each routine delivers consistently correct results in all cases – including across different processors, hardware architectures, programming languages, compilers, runtimes, standard libraries, and so-on.

Incredible amounts of work have gone into designing, coding and testing most high-end numerical libraries. What's more, the code has been reviewed by many individuals, including both practical and theoretical mathematicians. Generally speaking, you can be confident that the math is correct.

Beyond consistent correctness, there’s also efficiency. You’re not running that FFT or BLAS routine once; it’s being executed hundreds, thousands, millions of times, perhaps, during the execution of your program. Efficiency matters, including both raw speed but also use of resources like memory and threads.

An advantage of most numerical libraries is the tuning that goes into the code – a lot of hand-crafted C or Assembler code, in some cases. In other words, it’s fast. Increasingly, those libraries are also tuned for multicore processing. You could never justify spending the resources to do this yourself.

I had had experience with many numerical libraries, ranging from Intel’s Math Kernel Library to AMD’s Core Math Library to the IMSL Numerical Libraries to the NAG Numerical Components. They are all good, all recommended.

The tradeoff is that many numerical libraries are costly. If you need math, though, licensing one of the libraries is a bargain – and you can do the ROI calculations on a four-function pocket calculator.


With mobile, it’s all about the installed base

At the Google I/O 2012 conference, the Internet giant announced the availability for its Chrome browser for both Android and iOS devices – both tablets and handsets.

The next day, I was able to install the Chrome browser for iOS immediately onto an iPhone 4 (a device released in June 2010), an iPhone 3GS (released June 2009) and an iPad 2 (released March 2011).

Chrome for Android was also available immediately for the Google Nexus 7 tablet given to each attendee of Google I/O 2012. When I went to install it onto a Samsung Galaxy Tab 10.1 (released February 2011) it didn’t work. It also didn’t work on my older HTC Evo phone.

As a teenager would say, “You can run Chrome on a three-year-old iPhone but not a one-year-old Android tablet? Epic fail.”

Take a look at the requirements. In the Mac ecosystem, Chrome requires iOS 4.3 or later. That version of the operating system was released in March 2011, but could be user-installed onto older hardware.

By contrast, the Google Play store says, “This app is incompatible with all of your devices” and indicates that Android 4.0 (Ice Cream Sandwich) or above is needed. It’s the very rare machine that can be upgraded from Android 3.x to Android 4.x. And thus, you have the incongruity that a three-year-old Apple device can run Google’s feature-packed browser, but a one-year-old Android tablet (heralded as a showpiece by Google!) cannot.

This is going to hurt Google in the long run, if they continue to leave operating system upgrades in the hands of the device makers, and if they let hardware makers orphan handsets and tablets so quickly after they are released.

Microsoft, of course, is orphaning everyone who purchased a Windows Phone 7.x handset, because those phones won’t be upgradable to Windows Phone 8.x. However, I feel confident that this is a one-time break from the past. Also, Microsoft, like Apple – but unlike Google with Android – is keeping in control of the upgrade path.

It’s bad enough that Android devices offer different user experiences depending on which hardware maker you prefer. The look and feel of an HTC is light years from that of a Motorola or Samsung phone.

When even Google’s own apps won’t work on older machines, most of us lose. Who is the winner? The lack of backwards-compatibility and customized user experiences suits the needs of the hardware makers – not consumers, developers, or even enterprise customers. I’m sure they like this situation just fine.


Cisco and the undesirable consequences of automatic firmware updates

Harmless kerfuffle? Abuse of corporate power? Enablement of Big Brother? No matter what you call it, Cisco’s recent firmware updates to its Linksys home routers is troubling.

According to a story published on ExtremeTech by Joel Hruska, “Cisco’s cloud vision: Mandatory, monetized, and killed at their discretion,” Cisco pushed out a firmware update to some models of its LinkSys routers for homes and small businesses. One effect of the firmware update is to move administration of the routers from a local application to a service on Cisco’s Connect Cloud services.

This means that router owners must now sign up for Connect Cloud in order to manage their routers, but the Cisco terms of service for the cloud service give lots of power to Cisco.

Hrusksa’s story says that Cisco has changed the terms of service after a firestorm of customer complaints. As of July 5, they contain lots of clauses about the type of traffic that you can use on your home network. It also says,

You agree that Cisco may suspend or terminate your access to the Service without notice if (a) Cisco suspects or determines that you have violated this Agreement, (b) Cisco determines that your actions cause Cisco to be in violation of any agreement or policy needed to run the Service or (c) Cisco is required to do so by any court or government authority in any country.  You agree that Cisco will not be liable to you or to any third party for any suspension or termination of your access to the Service as a result of any threatened or actual violation of this Agreement.

Cisco may, upon such termination, deactivate or delete your account and any related data, information, and files, and bar any further access to such data, information, and files through use of the Service.  Such action may include, among other things, accessing your data and/or discontinuing your use of the Service and any and all rights granted to you in connection with the Service without refund or compensation. 

Note that if Cisco kicks you off Connect Cloud, you will not be able to administer your LinkSys router. While you might lose control of your router, Cisco doesn’t care about that issue. How about this section of the Cisco Connect Cloud Supplement to the Cisco Privacy Statement:

Cisco Connect Cloud software is updated from time to time to provide additional features, address technical issues, and generally make your user experience better. We may add to or upgrade the Service to provide you with new features on an ongoing basis. We may also make available new services in the future. New services provided by third parties or service providers will be governed by the privacy policies of the respective third party or service provider. The Service automatically checks for updates to the firmware/software to help keep your network running at a peak performance and provides alerts as to the latest firmware/software. The auto-update feature offers the ability to download the next available version in the background. Cisco Connect Cloud offers the auto-update feature by default, but you can change your auto-update options by changing your settings within Cisco Connect Cloud. By leaving the auto-update feature as a default, however, you will avoid disruption to your home network and overall Internet connectivity. In some cases, in order to provide an optimal experience on your home network, some updates may still be automatically applied, regardless of the auto-update setting.

In other words: You purchased the router, but Cisco may decide to push new software or change its functioning at any time – including installing third-party software without your knowledge or permission, or without giving you're the opportunity to review those third parties’ privacy policies. And of course, Cisco itself can change its privacy policy at any time.

Remember, we are talking about a network router here – something that sees every packet on a home or small business network. And Cisco is accused of helping the Chinese government build the “Great Firewall of China” to help it spy on its dissidents. What else might it do?

While automatic firmware updates are certainly convenient, the fact that you can’t turn them off is worrying. Personally, I wouldn’t buy a Cisco router. But who else can push firmware updates to your technology without your knowledge or permission? This, sadly, is the future.


Android and iOS advance, BlackBerry retreats, Windows Phone relaunches

The past few weeks has seen a lot of excitement in the mobile space. This past week we had Google I/O, where we got a first look at Android 4.1 “Jelly Bean,” which offers solid improvements. The previous week Microsoft unveiled Windows Phone 8 – a near-total relaunch that will excite future customers and ISVs, while disappointing existing Windows Phone 7.5 customers. This is comes on the heels of Apple’s World Wide Developer Conference, which highlighted iOS 6’s new social media integration, home-grown mapping engine, and faster browser engine.

All of these platform upgrades are due to ship in 2012.

We also heard from Research in Motion about a delay in its already-pushed-back BlackBerry 10 smartphone operating system. It’s now scheduled for the first quarter of 2013. The company also lost a ton of money in the past quarter. It’s reasonable to predict that BlackBerry is toast, or soon will be.

Let’s dig a bit deeper into the three, uh, more viable smartphone platforms – and what they mean for us.

Apple’s iOS 6 is merely an incremental upgrade. Apple has moved from Google’s mapping system to its own Maps engine, which is fully accessible via APIs. You can also integrate Facebook into apps, tap into the Reminders system, go closer to the hardware with the camera, and leverage an improved WebKit rendering engine.

The good news is that from the developer perspective, not much has changed; you get some new capabilities, but if you don’t need them, you don’t need to do anything.

Existing Apple customers won’t be standing in line at midnight to download iOS 6 – but they will appreciate its upgrades. That’s because Apple continues to win the award for best support of older hardware. All current iOS devices, including iPhone, iPad and iPod touch, should be able to run everything in iOS 6. Most features will run on the 2011’s iPad 2 and even 2010’s iPhone 4.

Microsoft’s Windows Phone 8, code-named Apollo, is a huge, huge, huge upgrade. Everything changes. It’s a real reboot, as the platform moves from the Windows CE kernel to the Windows kernel (either x86 or the new Windows RT kernel for ARM processors). For the first time, Windows Phone can run on multi-core hardware and use screens larger than 800x480. Because the WP8 kernel is the Windows 8 kernel, WP8 phones should work with any Windows 8 device driver. That should let you do some pretty interesting hardware integrations – and gives WP8 a huge advantage of iOS and Android.

Near Field Communications (NFC) is a big part of WP8, as it is with Android. iOS still doesn’t get NFC – but I expect Apple to catch up fast, perhaps with the next-generation iPhone and an iOS 6.x update.

Most importantly for apps developers, you can also write apps in C/C++ or even use the .NET framework with C# or Visual Basic. That truly enables code migration to the mobile platform.

The bad news is that WP8 will require all new hardware. The only upgrade for existing WP7.x users (including those who just bought the much-hyped Nokia Lumia 900 phone)  is a cosmetic upgrade called Windows Phone 7.8 that improves the Start Page with new colors and three sizes of Live Tiles. Beyond this small enhancement, every existing Windows Phone customer is out of luck until their carrier’s contract lets them buy a new phone. Ouch.

Google’s Android 4.1 Jelly Bean is another incremental upgrade, similar in scope to what Apple is doing with iOS 6. The biggest change is called Project Butter, which improves the dreadful synchronization of the touchscreen with screen composition/refresh. This, along with improved triple buffering should go a long way toward eliminating common complaints that the Android UX is sluggish, unresponsive or even buggy, when it’s simply out of sync.

Another area of improvement is notifications – where you can provide more info to the end user, and even let them respond without switching to your app. You can more easily communicate with devices connected via USB or Bluetooth, access the NFC stack, and work with a better HTML5 renderer. A popular feature should be an upgrade to Ice Cream Sandwich’s message queuing service. Called Google Cloud Messaging, the system can now deliver up to 4K of data – and can leverage multicast.

We don’t know which existing Honeycomb (Android 3.x) and Ice Cream Sandwich (Android 4.x.x) devices will run Jelly Bean. This is determined largely by each hardware maker on a model-by-model basis. If past experience is any guide, some existing Android handsets or tablets will be upgradeable to Android 4.1, but most will be orphaned.

That’s a shame — but it’s better than those poor Windows Phone 7.x owners can expect.

About Me

My Photo
Co-founder and editorial director of BZ Media, which publishes SD Times, the leading magazine for the software development industry. Founder of SPTechCon: The SharePoint Technology Conference, AnDevCon: The Android Developer Conference, and Big Data TechCon. Also president and principal analyst of Camden Associates, an IT consulting and analyst firm.