Reflection on Best Practice Software Engineering

In this post I ask and answer 5 quick questions about software testing that any software engineer should know. Try to answer the questions before looking at the answers!

1. Why are coding standards and tools like Checkstyle important? Is running Checkstyle an 'expensive' operation?

Coding standards help by enforcing consistency that makes code easier to read and understand across an organization. Enforcing standards saves time by allowing developers to spend more time coding and less time formatting it. Once the standards are agreed upon, tools like Checkstyle can be run often, from a build script, with almost no 'cost'.

2. Why is "exhaustive" testing difficult?

There are too many possible program inputs and outputs to 'exhaustively' prove that a program is "correct".

3. What is equivalence partitioning?

Equivalence partitioning divides all possible input data into classes or partitions that represent each input 'partition', such that testing one value is equivalent to testing all values in that 'partition'. For example, if the input is a number, a test design heuristic might suggest the following test values: typical, empty or null, illegal, minimum, and maximum, too small a value, and too large a value.

4. What type of errors cannot be detected with white-box testing?

Errors of omission, i.e. code that should have been written - missing exception handlers, missing conditional tests, missing tests for validating command line argument formats.

5. What is the difference between validation and verification testing?

Validation testing basically tests that "we are we building the right software to get the job done" and verification testing tests that "we are building it correctly". In addition, validation testing involves the customer, and verification testing involves the code-base.

Configuration Management with Subversion and Google Project Hosting

CM using Subversion

It is almost impossible to build large, scalable, collaborative software systems without a (good) configuration management system. We can find evidence of this as the demand for better CM tools over time has driven the availability and adoption of more robust and easier to use CM systems. Some of these are relatively new - like Git - and are gaining popularity quickly, with the promise of speed, reliability, security, and a host of other features.

Any Configuration Management approach should address the three classic CM problems:

• The double maintenance problem (must prevent multiple copies of the same file)

• The shared data problem (must allow developers to work on the same file simultaneously)

• The simultaneous update problem (must allow the developers working on the same file to update it without clobbering each others' changes)

Subversion solves these problems by using file versioning, file checkout and commit, and resolving commit conflicts using merging. In addition there are many more features that I won't go into in this post, but I encourage you to check out on the Apache Subversion site.

For my local Subversion client, I used the Tortoise SVN client based on Subversion 1.7, which integrates seamlessly into the Windows 7 environment using context-sensitive menus. It decorates files and folders under Subversion control with easy-to-understand icons.

Using Tortoise SVN, it was very easy to put my project (source tree) under Subverion control, make my initial commit, and thereby place my project into the repository on Google project hosting. Almost immediately I realized I needed to delete some files I really didn't want in the repository, and again, this was surprisingly easy to do.

Google Project Hosting

Project Hosting using Google Code provides an easy to understand and easy to use environment for collaborative development of open source projects. It was a piece of cake to create a new project backed by Subversion for the repository. My open source project hosts my competitive Robocode robot code. It provides issue tracker, wiki pages, admin, download, and other useful pages/links on a per-project basis. I was able to document changes to the repository, view revisions/changes, add 'commit' access for project collaborators, and create both a UserGuide wiki and a DeveloperGuide wiki using the builtin editors.

I accomplished all tasks I set out to do in this first attempt at open source project hosting using a Google project hosting and a local Subversion client.

I had a few moments of misunderstanding when .svn folders were not appearing in my Tortoise SVN client. By using a shell window, I made sure they weren't there and it was not a Tortoise SVN issue. However I was able to checkout files, make changes and commits with correct expected results.

I learned from the Tortoise SVN forum (registration required to post) that Subversion 1.7 no longer maintains a .svn directory for each subdirectory of a project's files. Instead it uses a single .svn directory located at the project root. I verified this on the Apache Subversion site.

I was surprised when I was able to checkout files from the repository without supplying any credentials. I soon discovered that Google Project Hosting asks for credentials when comitting changes, not when checking out files from the repository. And Tortoise SVN can store those credentials resulting in an even quicker, more seamless workflow.

Robocode, Automated Build Systems, and Automated Testing

 Overview

As an exercise in developing systems that scale, I continued development of a competitive Robocode robot, integrating it with an automated build system that builds the system, runs the robot simulation, and tests the code using JUnit tests, acceptance tests, behavioral tests, and quality assurance tools. This build system uses a collection of Ant scripts; coding is still done using Eclipse. The big difference is that now, with a few simple commands from a shell, I can accomplish all the tasks outlined above. The real benefit is that as the code-base grows, the same scripts can be run. Of course, new tests and script modifications can be added, but the impact of maintaining a few scripts or adding tests should be negligible compared to the time savings if I was doing all the build, run, test, and  quality assurance manually.

The Robocode “Surfer” Robot

Movement: My strategy for moving the Surfer robot is to conserve energy and perhaps fool tracking robots. Surfer follows a counterclockwise circuit of the right half of the battlefield, then reverses its direction and follows a clockwise circuit. Additionally, Surfer conserves energy by not firing often, and varying the bullet strength.

Targeting:  Surfer stops and rotates it’s radar to scan for other robots at 4 places during each clockwise or counterclockwise circuit: top center, top right corner, lower right corner, and bottom center.

Firing: Any time Surfer scans an enemy robot, it fires at it with a bullet strength based on both the distance from the enemy and the size of the battlefield itself. Surfer also fires more bullets (up to 3) if it is closer. The thresholds are: fire 3 bullets if 200 pixels or closer, fire 2 bullets if 300 pixels or closer, or just fire 1 bullet when farther away.

Results:  Here are the results of 5 battles of ten rounds each against some sample robots:

Walls: Surfer wins 0/5.

RamFire: Surfer wins 5/5.

SpinBot: Surfer wins 1/5.

Crazy: Surfer wins 5/5.

Fire: Surfer wins 5/5.

Corners: Surfer wins 3/5.

Tracker: Surfer wins 5/5.

SittingDuck: Surfer wins 5/5.

So Surfer can reliably beat most of these sample bots except Walls, SpinBot, and Corners. Given Surfer’s conservative nature, I think these results are pretty good. To beat Walls, SpinBot, and Corners, I think Surfer needs a more advanced targeting algorithm and perhaps a more advanced strategy for bullet firing.

What I would improve in the Surfer robot initially, includes locking the radar once an enemy is scanned as well as devising a strategy when colliding with another robot. Beyond that, after becoming more familiar with the game physics and the Robocode API, literally dozens of changes and improvements could be made to Surfer.

Testing

My testing approach begins with JUnit tests to test the methods I use to calculate the number of bullets to fire, as well as the bullet strength. I also perform behavioral tests that check if my robot visits the positions on the battlefield that I expect it to, and that the bullets it fires vary in strength. My acceptance tests are implemented by having my robot battle against other robots. In addition I used Jacoco to get feedback on "coverage". This is very useful because it reveals lines of code are not being executed during testing, and if they are not executed, they may well contain bugs.

Quality Assurance

Above and beyond JUnit, behavioral, acceptance tests, and Jacoco, QA was done using checkstyle, findbugs, and PMD. Although these three products have some degree of overlap, they can find many kinds of errors, including 'best practice' coding style errors, unused variables, code that can never be reached, etc... Once they are built into the automated build system they pay big dividends that increase as the project grows and the lines of code increase. To do these type of tests manually would be painstakingly difficult and time consuming.

Lessons learned

From this project I learned that running automated testing and QA tools can be done in matter of seconds, and that these tools will scale to larger systems where manual techniques will not scale. Bugs, incorrect behavior, and errors in coding style, can be discovered more quickly than performing these processes manually - by perhaps an order of magnitude or more. Virtually no large software development effort would be undertaken without tools and techniques similar to the ones I used. Even though the QA tools may take some time to configure and understand, and scripts may take some time to develop, in the long run they will save an enormous amount of money, time, and effort.

Build Systems and Apache Ant

In this Blog entry I write about my experience learning about build systems in general, and Ant in particular.

Ant Code Katas

Again the "Code Kata" approach deserves praise for its role in learning. Until the last Ant Code Kata (#8) the incremental Kata approach proved itself to be appropriate for learning as success came easily with Katas 1-7, motivating me each time to move to the following task. Here are the simple Ant build scripts I implemented:

1. Ant Hello World (Ant echo task)
2. Ant Immutable Properties (Ant property task)
3. Ant Dependencies (Ant target dependency resolution)
4. Hello Ant Compilation (Ant javac task)
5. Hello Ant Execution (Ant java task)
6. Hello Ant Documentation (Ant javadoc task)
7. Cleaning Hello Ant (Ant delete task)
8. Packaging Hello Ant (Ant zip task, ant copy task)

Learning Ant

I was generally impressed with the wide range of useful tasks that can be accomplished by scripting using Ant. By far the most interesting Kata was the zip task, which required understanding the zip target: name the archive, where to place it, and more generally, learning how to include and exclude files. There are also tasks for setting and referencing a multi element classpath. Ant has several ways to do this. Ant properties are immutable, meaning once set, they can not be changed. Integration with Eclipse was expected: Eclipse has a colored syntax editor for working with Ant scripts, Ant "Run Configuration" editor, a "Run as Ant build" button, and it diplays the value of Ant properties when hovering over them in the editor with your mouse.

Automated Build Systems

I learned that automated build systems offer a low cost way to build software releases, including version(ing), automated running of tools such as javadoc, checkstyle, and findbugs, and can even download tools and libraries. They do not interfere with the developers' choice of IDE. An automated build script can be used to automate and verify the build often, with almost no cost once the script is written. Although build systems are not a substitute for JUnit tests or software reviews, practically every substantial software system uses some sort of automated build system to increase productivity, efficiency, reduce errors, and save time.

Robocode Code Katas

Robocode

Robocode is a hugely popular programming game, or more technically, a battle simulation engine, available freely as open source. However, you don't have to be a seasoned game programmer or Java guru to experience or enjoy it. Robocode is easy to install, configure, learn, and understand, even for beginning Java (or .NET) programmers. There are a wealth of websites that contain tutorials, examples, and even complete code for both simple and complex robots.
Using the Robocode API, you can develop robots (or more correctly: tanks) to do battle against other robots of your own, or sample robots provided by the Robocode game itself. The idea behind Robocode is to learn more about Java and programming, and of course, both gaming and Robocode itself, and have fun while doing so. There are even competitions for those that are interested.

Fig 1. Robocode Battle with four robots.

Robocode Code Katas

A "code kata" is an exercise in honing your skills as a programmer based on the concept of katas in karate. Kata are exercises - you repeat each exercise many times, making incremental improvements each time, and the complexity of the exercises increases as you progress through them. The learning discipline of code katas is similar. For more about code katas, the concepts are explained nicely in the blog articles I read by Jeff Atwood here and the more in-depth treatment by Dave Thomas here .

Back to Robocode! I developed a handful of simple Robocode robots in the spirit of code katas, that enabled me to become familiar withe Robocode game engine, the Robocode programmer API, and to incrementally increase my skills as a Robocode/Java programmer. This effort also included learning more about the Eclipse IDE, as well as adhering to a published set of development guidelines. This included following a specification for Java code formatting and coding style, as well as producing professional-level documentation using Javadoc. The book Elements of Java Style  provided most of the style and format conventions I used in order to develop code that is easy understand and maintain.

So as an introduction to learning using the code kata method, I implemented the Robocode robots described below. Each successive 'exercise' was a little more complex and more challenging than the previous one.

• Position01: The minimal robot. Does absolutely nothing at all.
• Position02: Move forward a total of 100 pixels per turn. When you hit a wall, reverse direction.
• Position03: Each turn, move forward a total of N pixels per turn, then turn right. N is initialized to 15, and increases by 15 per turn.
• Position04: Move to the center of the playing field, spin around in a circle, and stop.
• Position05: Move to the upper right corner. Then move to the lower left corner. Then move to the upper left corner. Then move to the lower right corner.
• Position06: Move to the center, then move in a circle with a radius of approximately 100 pixels, ending up where you started.
• Follow01: Pick one enemy and follow them.
• Follow02: Pick one enemy and follow them, but stop if your robot gets within 50 pixels of them.
• Follow03: Each turn, Find the closest enemy, and move in the opposite direction by 100 pixels, then stop.

Four additional robots I have not implemented yet but would like to, are:

• Boom01: Sit still. Rotate gun. When it is pointing at an enemy, fire.
• Boom02: Sit still. Pick one enemy. Only fire your gun when it is pointing at the chosen enemy.
• Boom03: Sit still. Rotate gun. When it is pointing at an enemy, use bullet power proportional to the distance of the enemy from you. The farther away the enemy, the less power your bullet should use (since far targets increase the odds that the bullet will miss).
• Boom04: Sit still. Pick one enemy and attempt to track it with your gun. In other words, try to have your gun always pointing at that enemy. Don't fire (you don't want to kill it).

Discussion

This exercise taught me many things. The Robocde platform, code kata approach, together with the development standards, and Eclipse IDE, was indeed a fun way to practice, learn, and improve Robocode, Java, and Eclipse skills. I was surprised to learn how quick and easy it was to download, install, and configure the game engine, and to code and run my first robot!

The Robocode API was mostly self explanatory, but there were some exceptions. I used Google to search for examples of calling Robocode methods in the context of implementation examples to overcome this. The API seemed abundant enough to build robots quickly and intuitively, but it wasn't long before I was wishing more methods were available, for example, a goTo(Double x, Double y) method. Looking forward, I believe that it would be difficult and time consuming for me to build a very competitive robot (the kind that compete in the Robocode competitions). As I am still building very rudimentary robots, I don't yet have a strategy to build a competitive one.

I believe the code kata approach is a valuable tool for learning. I feel we have all used this approach at some time or another, to some degree, when the opportunity is there (e.g. enough time to use it, no conflicting requirements). I don't know if there is empirical evidence that this approach to learning is the best one, but in an appropriate scenario I consider it a valuable approach. For this endeavor - learning Robocode for fun and exploration - it felt like a productive and appropriate technique.

Once I understood the game engine and it's seamless integration with Eclipse, my difficulties were mainly with the trigonometry required to build robots within the time frame I set aside for this exercise. Having long-forgotten most of the formulas needed, and knowing where and when to use them without hesitation, slowed my ability to code the robot behaviors. I was able to look up trigonometric equations and formulas on the internet, using mainly this site suggested by a friend.

FizzBuzz

FizzBuzz with Eclipse

After reading the blog posts by Jeff Atwood here and by Scott Hanselman here, I implemented the FizzBuzz program using Eclipse in about 7 minutes - including coding a
junit test class. Since I knew a solution ahead of time, the bulk of the time was spent typing, rather than devising an algorithm. Using junit for the first time, I used control+space in Eclipse to see the available methods in the org.junit.Assert class. When I forgot to use String.valueOf() when dealing with integer values (the method return type is String), Eclipse flagged this error in real-time as I was typing. I was able to correct it immediately rather than wait for a compile time error to realize my mistake had I been using an ordinary text editor and javac.

FizzBuzz.java





FizzBuzzTest.java



I implemented an alternate FizzBuzz algorithm by adding another method to my FizzBuzz class and tested it using junit as well. I used "refactor code" in Eclipse and all the method names in the junit test class were changed for me automatically. In addition I used "window->preferences" in Eclipse to change the style of the braces and add blank lines to the code automatically while typing as well as when using "source->format" from the context menu.

Summary

I learned some valuable lessons by reading the blog posts mentioned above and coding the FizzBuzz program with Eclipse. It's a good thing to be able to commit a working knowledge of correct Java syntax to memory, even though it's easy to look at or cut-n-paste existing code, use the API documentation, or search for code examples using Google. With so many Java classes in the API it's not possible, or even time well spent, to memorize every class! However, the basic syntax, such as what's needed to create the FizzBuzz program, is a requirement for any programmer taking his profession seriously. I also gained some insights into the mindset of interviewers, and feel that becoming an expert Eclipse user will be a very valuable skill, increasing my productivity and efficiency when coding.

Reflection on ICS 613

Based upon our assignments and readings to date, I feel ICS 163 will benefit us by introducing and/or reinforcing concepts and skills that are essential to being a successful software engineer.

Open Source Software and the Three Prime Directives

Open Source Software and the Three Prime Directives

Overview: 7-zip

7-Zip is the package I chose to download, which is a file archive pack/unpack utility supporting an everyday task for most software professionals and even end users.  Its stated objectives include a high compression ratio and support for many/most file archive formats encountered on modern computer systems. Additionally it is claimed to run under most flavors of Windows as well as Linux/Unix.

7-zip can be downloaded at:        http://sourceforge.net/projects/sevenzip

Prime Directive 1: The system successfully accomplishes a useful task.

Given the overview above, it is obvious that the functionality promised by 7-zip would be both useful and beneficial. Therefore it qualifies as a utility that provides an extremely useful everyday task for a broad range of computer users, since a large number of downloaded files are packed, and conversely there is a need to pack files and/or directories before emailing or uploading them to other users.

 

What other useful functionality is provided? Aside from the basic tasks of packing/unpacking archives, 7-zip provides features such as encryption, self-extracting capability, integration with the Windows shell, a file manager, a command line version, and localizations for 79 languages.

So the system satisfies PD1: successfully accomplishing a useful task.


Prime Directive 2: An external user can successfully install and use the system.

Achieving Success with 7-zip was easy. One click to download, run the downloaded .exe file to install it, and simply choose the install directory was all that was required.

I was able to use 7-zip immediately with no instruction, readme file, release notes, or documentation.

I especially liked the seamless integration with Windows context menus. Intuitively, I right-clicked on a .rar file. I was presented with a menu that included several 7-zip options including open archive, extract files, and test archive.

I chose 'open archive' and a well-thought out and simple UI containing the .rar file was presented with just a few buttons, as well more options contained in a menu bar across the top of the application. All I needed to do was click the extract button and provide a location, and the archive file was unpacked in less than a second.

 

7-zip satisfies PD2: I was able to successfully install and use the system.

Prime Directive 3: If you were going to attempt to modify and improve the system, how easy would it be for you to do so?

The package has it’s own home page at http://www.7-zip.org, and another developer-specific page at http://sf.net/projects/sevenzip/develop.

Not only did the developer page include a link: "Send a request to join this project" but the project home page showed statistics for Bugs, Support Requests, Patches, Feature Requests, and Public Forums.

Each statistic had a link to very detailed and easy to understand data arranged in a table format. In fact, the public forums page had both a forum for help, and a forum for discussion including a useful search by keyword feature.

In addition, the 7-zip project home page contained license information, version change history, and a (favorable) compression ratio comparison chart comparing some of the leading popular archivers currently in use.

Without looking at the source code and other developer documentation (available by request only) I cannot be sure PD3 is satisfied. If the source code and developer documentation is comparable to the rest of the package, I suspect that it would be easy to modify and improve the system, thus satisfying PD3.