Category: Technologist

This category contains posts related to technology and my work as a technologist.

Web Searching Can Be Dangerous For The Novice Developer

February 19, 2011 by Sherwin James | 0 comments

With the vast amount of information available on the web today, it’s no wonder that most developers use it as the first go-to source for coding assistance.  The entry points may differ (e.g. Google search, Bing, stackoverflow, or others), but the intent is usually similar (I am stuck on a technical problem; I don’t know where to start; I would like to expand my knowledge on …).

That’s all well and good, but, for the novice developer, this may set a dangerous pattern of usage.  For much of what we get when we search the web for solutions are pieces of information without much context – or with widely varying context.  Gone are the days when the “how-to” book, or the language reference guide, or the system manual is the first item reached for when assistance is needed.  Those items tend to provide much more context for usage than the fragmented responses from web searching, but they often required a bigger investment in time and patience.

I recall when I first got into programming that I always had my “The C Programming Language – Second Edition” by my side, as well as my handy “vi Quick Reference Guide” and man pages for Korn shell scripting.  They were extremely valuable assets that provided the proper context I needed to help me understand which solutions worked best under which conditions.  I believe that having spent my software development adolescence  during the time of contextual references and how-tos helped to established a pattern of thinking that promotes solution composition, as opposed to solution replication.

Don’t get me wrong, there is amazing value in the collective knowledge of the web.  But picking the most appropriate parts out of that massive information repository, and then using it appropriately, takes some thought and becomes easier with experience.  It also helps to have a good grounding on how to think and apply knowledge.  There are strong temptations to apply a cut-and-paste workflow when applying web search results to a problem, that might look something like this:

have a problem -> search for problem -> get responses -> apply response 1 -> fail now -> apply response 2 -> fail now -> apply response 3 -> fail much later … -> rethink whole approach.

Instead I suggest a different approach, which involves some additional analysis of the problem and results.  It might go something like this:

have a problem -> determine desired solution outcome -> determine affected areas -> search for affected areas -> get responses -> analyze affect of responses -> chose effects that best align with desired solution outcome -> tweak/re-factor as necessary -> succeed a little later.

This approach may take a little more time, but should produce more resilient solutions in the long run.

Is it important to know Why we are implementing something?

June 1, 2010 by Sherwin James | 2 Comments

When given a task to implement an application, or other software technology, most developers / development teams first ask the question “What are the requirements of the application that is to be developed?” (some even jump straight to “How to develop?” – but that’s a topic for a different discussion).  I believe that the first question, or at least the question right after “What“, should be “Why is this needed / desired?”.

Yes, understanding who, what, when, where and how are important (or even critical) to the successful implementation of a software product (or any other activity, for that matter).  But while the answers to these questions will most likely provide the quantitative controls for an implementation project (e.g. specific requirements, users and user types, delivery deadlines, internal vs. external usage, etc.) – the question of “why” seeks to answer the more qualitative ones.  The answers to why (when examined by someone of some level of experience) can provide valuable info on purpose, priority, constraints, dependencies, biases and other such matters.  It can lead to such decisions as whether the task should be undertaken in the first place; whether it would be better to buy a product, instead of build one; whether the development team should spend more time on making the UI top-notch vs. building a completely open and extensible back-end.  These insights, and much more, can result from asking “Why?“.

So when next faced with an implementation task, don’t be afraid to ask the sometimes difficult question of “Why?“.  It may save you a tremendous amount of effort in the pursuit of what and how, as well as the other questions of choice.

The Evils That Oppress Our Apps

March 15, 2010 by Sherwin James | 1 Comment

In a perfect world, we would create applications that run with limitless speed, are completely open, and gets the job done exactly right the first time, each and every time. Well, this is obviously a fairy tale, because we live in a world full of evils that aim to oppress our apps. I’ll attempt to describe some of those evils, including their main sadistic traits, and also attempt to describe some of the weapons you can give to your apps to help combat these evils.

Evil #1 – The Blob

It’s MO (modus operandi):

The Blob aims to bring our apps to an unusable crawl. You can find it lurking in many places – nestled in between the gaps and interfaces in our code; stuck to the walls and floors of our servers; and especially oozing all over the information super-highway, causing traffic jams at the most inconvenient times. It’s a viral beast, that hitches rides on our apps (especially the unarmed, defenseless ones), and leaks a little bit of itself behind as our apps travel to/from its poor users, who just want to get stuff done.

Weapons to fight The Blob:
  • A Strong Framework For Your App – Put some up-front effort into the design your application that makes use of proven design patterns. Take advantage of those who have fought The Blob before, and have incorporated those lessons learned in their frameworks and designs.
  • Caching – Store and manage replicas (in whole or in parts) of your application data, close enough to your users to gain the most efficiency, but far enough away to benefit from shared access among users. The less distance the users have to travel to get data – the less opportunity The Blob has to get in the way.
  • Performance Testing – Don’t short-change testing of your application for performance inefficiencies before release. Best to find the unnecessary gaps and crevices in your apps where The Blob may hide, and refactor them away.
  • System Maintenance – Keep your servers, workstations, and other hardware and system software up to date with latest (or near latest) fixes and versions. Also, ensure that your hardware is packing enough power to match the needs of your app. The Blob loves slow, deprecated and overworked equipment, so don’t indulge it.

Evil #2 – The Trickster

It’s MO:

The Trickster is a deceptive villain indeed. It will stoop to any level to gain access to your application information, in order to use it for its own malicious purposes. The Trickster’s bag of tricks include identity theft, mis-direction, denial-of-service (blocking out the good guys), using your app as its own weapon, and many of gimmicks. Its goals range from being an annoyance to downright destruction, with profit and notoriety somewhere in between.

Weapons to fight The Trickster:
  • A Strong Framework For Your App – This same weapon used against The Blob, can prove to be valuable against The Trickster as well. A strong framework narrows the places in which the The Trickster can play, bringing it into the light and exposed for better detection.
  • Adherence to Security Standards/Practices – There are a plethora of standards, practices and guidelines that can be followed to help ensure a secure application. There is no 100% guarantee against a security attack (other than not building the app), but your security efforts should match your application needs for secure operation. In defining, designing, building, testing and deploying your application, keep mind that The Trickster is always lurking, so always be on the defensive.
  • Secure Data At The Source – Protect your application’s data as close to the source as possible, while still maintaining the level of functionality required. This includes minimizing the data retrieved by your app to what is really needed a particular point in time. The less opportunity that The Trickster has to hijack your data in transit, the better. Unfortunately, some of the weapons (like this one) that work well against The Trickster tend to benefit The Blob – so you’ll need to clearly understand your priorities in the overall war against these villains.
  • Defined API and Unit Testing – Clearly define the API (input and output – including exceptions/errors) for each of the sub-components in your application. Verify each sub-component with Unit Tests to ensure your app supports what is needs to support/enable and nothing else. Check out my post on this topic – How the API and the Unit Test Can Save the Software Industry. The Trickster loves to take advantage of loosely defined/implemented interactions between components in your app – well defined APIs help to enforce the rules for all, including the Trickster.

Evil #3 – The Garbler

It’s MO:

The Garbler is a master of confusion. You never know it’s there, until it’s too late to do anything about removing it. The Garbler inserts itself between you (the application implementer) and those responsible for providing requirements for your app – and proceeds to mangle and confuse the communication in-between. It sometimes even makes it way between peer developers. The Garbler’s main objective is to disrupt communication enough that the application turns out to be completely different from what was required/intended, sometimes forcing the creation of utter garbage that is full of bugs.

Weapons to fight The Garbler:
  • An SDLC Process – By far the best way to fend off The Garbler is to follow a Software Development Life Cycle (SDLC) process. Doing so will greatly help to solidify and standardize the communication between roles on any software project, as well as help ensure that the proper steps are taken (and deliverables produced) that moves the application forward towards a quality product. With an SDLC in place The Garbler has little opportunity to inflict confusion into the process.
  • Completed Functional and Acceptance Testing – As an important safety net, functional and user acceptance testing provides key opportunities for the effect of The Garbler to be corrected and cleaned-up. Testing should happen often and should not be considered as an optional step. The Garbler is probably the oldest and most experienced of the evils, and it is quite persistent – so take advantage of all opportunities to prevent it from doing its dirty work. It’s unlikely that you can completely stop it, but every attempt should be made to minimize its effect.



It is important to know your enemy – these evils sure do know your application. There are other villains out there, but these three are probably the most notorious. With the weapons mentioned, you can arm your app for battle.

Now go forth and fight the good fight!

The ABCs and XYZs of IT

February 7, 2010 by Sherwin James | 0 comments

There exists an alphabet soup of acronyms and abbreviations within Information Technology. Included here are some of those that are important for any IT technologist to at least be aware of – but better yet to understand with some depth.

  • AJAX – Asynchronous JavaScript and XML
  • ALM – Application Lifecycle Management
  • API - Application Program Interface
  • BI – Business Intelligence
  • CSS – Cascading Style Sheet
  • ERD – Entity-Relationship Diagram
  • ETL – Extract, Transform and Load
  • HTML – Hyper Text Markup Language
  • HTTP – Hypertext Transfer Protocol
  • LAMP – Linux, Apache, MySQL, PHP/Perl/Python software bundle
  • MDM – Master Data Management
  • MVC - Model/View/Controller
  • ODS – Operational Data Store
  • OOD – Object Oriented Design
  • POJO – Plain Old Java Object
  • REST – Representational State Transfer
  • RIA – Rich Internet Application
  • RPC – Remote Procedure Call
  • RSS – Really Simple Syndication
  • SaaS – Software as a Service
  • SDLC – Software Development Life Cycle
  • SEO – Search Engine Optimization
  • SLA – Service Level Agreement
  • SLA – Service Level Agreement
  • SOA – Service Oriented Architecture
  • SOAP – Simple Object Access Protocol
  • SQL – Structured Query Language
  • SSO – Single Sign On
  • TDD – Test Driven Development
  • UML – Unified Modeling Language
  • URI – Uniform Resource Identifier
  • WSDL – Web Services Description Language
  • XML – Extensible Markup Language

If you are not familiar with these, I strongly suggest you look them up. Google, Bing or Yahoo will do as a start.

Consistency Beats Correctness, Any Day

February 5, 2010 by Sherwin James | 0 comments

Let me start by stating the obvious: if I can have both consistency AND correctness in my technology dealings – I’ll take it. Unfortunately, that is seldom the case. Also, some definitions are needed to properly set the context for this discussion. For consistency, the meaning is fairly straightforward:

From: answers.com
1. Agreement or logical coherence among things or parts: a rambling argument that lacked any consistency.
2. Correspondence among related aspects; compatibility: questioned the consistency of the administration’s actions with its stated policy.
3. Reliability or uniformity of successive results or events: pitched with remarkable consistency throughout the season.

Correctness, however, is not so easy to pin down. Let’s take a look at a definition of correct (as an adjective):

From: answers.com
1. Free from error or fault; true or accurate.
2. Conforming to standards; proper: correct behavior.

The problem is, correctness – like truth – is relative and not absolute. What is correct for one, may be completely incorrect for another. So the best we can hope for is that something is appropriate for a person or group at a particular time or period. Standards help a bit to settle down this variance, but they have a long, long way to go.

So, with that context established, if I had to choose between consistency and correctness, I would hands-down choose consistency. A big part of being successful on any technology project is being able to predict the outcome with some measure of accuracy – whether its knowing the effort for each code component, system upgrade, hardware device, process change, or any other work item. That predictability is directly related to the consistency of the work items that must undergo change. Whilst the correctness of the existing targets of change has some impact on the effort for change, it doesn’t come close to the impact of having to apply multiple paths of change due to inconsistent target implementations.

Another aspect to consider is the consistency of the solution, as compared to the correctness of individual elements of the solution. An extensive amount of effort may be exerted in order to find and implement the correct individual solutions for each target of change, which can be of little benefit – or even be a detriment – to the successful and timely completion of a technology project. Often it is best to have an overall solution that is consistent across all or most targets, but may not be necessary totally correct for all targets of change.

How the API and the Unit Test Can Save the Software Industry

December 30, 2009 by Sherwin James | 2 Comments

While more established engineering disciplines, such and civil and electrical/electronic engineering, has had a couple of centuries to grow, fail, improve and stabilize – software engineering has only been around for a number of decades.  It’s akin to an adolescent teenager going through some dramatic growth spurts.

Software engineering, as a discipline, is expanding at a tremendous rate.  Unfortunately there is still much chaos with regards to standards, interoperability , and quality.  Part of what is fueling this chaos is the equally (or even more) dramatic growth of computing hardware [Check out Moore's Law regarding the doubling of transistor capacity on integrated circuits every two years].  It’s too easy these days to adopt the “throw-hardware-at-it” approach to solving software issues, instead of investing time to make the software engineering  part of the solution more efficient, standardized and qualified.

I believe that two key elements that will help to save the software industry from inevitable [yes, I can predict the future] implosion are the API and the Unit Test.  Wikipedia [I know it is not an "official" source, but it will do for this purpose] offers this introductory definition of API:

An application programming interface (API) is an interface implemented by a software program to enable interaction with other software, much in the same way that a user interface facilitates interaction between humans and computers.

I would extend that by saying that an API also establishes a contract [and yes,  I wish it were a legal contract, accompanied by certifications, penalties and expulsion - but I digress] between the consumer and software component that defines and enforces the following:

  • Purpose – A statement of intent for the software component, including a listing of contracted actions.
  • Inputs – The definition (including type and value constraints) of the information/data needed by the software component to perform the contracted action(s).
  • Outputs – The definition (including type and variability/tolerances) of the information/data produced by the software component as a result of performing the contracted action(s).  This includes any notifications (events, warnings, errors, etc.) that may occur during the process.
  • Changes – This can also be classified as output, but specifically defines the expected changes to the state of the software component and its environment (context) as a result of performing the contracted action(s).

I would describe a software component as a packaged (however crude that ma y be) set of software elements that is delivered (or sold) to a consumer.

With the establishment of a well-defined API as a “necessary and required” part of all software components, software engineering can begin to take its place among the other stabilized and mature engineering disciplines.  Consumers, whether they be end-users or other software engineers, will begin to better predict software behavior and have increased capability with integrating software components to meet their needs.

But wait… How can software engineers gain the confidence needed to guarantee their software components’ API contracts?  Well, testing is the answer – specifically unit testing.  Not to take away from the other types of testing (e.g. system, performance, security, etc.), but it all starts with a deliberate and specific unit test.  There are quite a few testing approaches in the industry to choose from to adopt (e.g. TDD – Test Driven Development), but it all comes down to ensuring quality and verifying expected results of software, as defined by the API.  In the case of unit testing, quality is verified from the bottom up.

So, today, what we can all do to advance software engineering in the right direction is to start with a well-defined API, and then build and Unit Test our software components to meet that API.  You never know – one day you may have to cough up some dough if you breach that contract.

Where Concepts Meet Reality

December 26, 2009 by Sherwin James | 1 Comment

As technologists , whether you are an architect, designer or developer, we often spend much time in that place where the concepts, that we believe to be important, meet the reality of the project or software development task that we need to complete.  That place can be one of enthusiasm and gratification, but it can also be filled with stress, indecisiveness and delays.

In my experiences I have found that having a set of key principles and guidelines with me as I travel through that place can be extremely helpful.  Here are a few of those:

  • Software development is not religion – One should take pride in their software development efforts, but avoid the temptation to follow the “rules” blindly.  The most important part of a technologist’s toolkit is their ability to think within the context of their environment.  Use the “rules” as landmarks and suggested paths as you navigate through that place where concepts meet reality.
  • State vs. Behavior – Understanding the difference between these two fundamental object oriented design concepts is important.  In addition, knowing how to separate the two (if needed) is crucial to solving a range of technical challenges.  In a true OOD implementation, an object encapsulates both state (properties) and behavior (methods).  In a services based implementation, state is held by the entity objects while the services handle the behavior.  I have found that it is often more practical to have a mix of the two within a software implementation, with groups (or categories) of objects conforming to the different approaches.
  • Build in and “out” – Try as we might to handle ALL  scenarios or events that could occur with the use of software,  we will inevitable fail.  Software design is best served, from a practical sense, when it attempts to be structured around those scenarios and events that are well understood and predicable.  However,  to handle those unexpected or rarely occurring events, the design should have an “out” or a “catch all” that can deal with those events in a generic manner.
  • Separate but together – When it comes to expandability and maintainability, having separate and distinct architectural layers in your software design can really help.  Traversal of layers in a contiguous order is also important, but sometimes we must forge non-standard paths to deal with particularly thorny issues (such as performance and compatibility).   But before we veer away from the true path, we must be aware of the consequences and be prepared to deal with them.
  • Test, test and test – Test early and often, late and big, or somewhere in between – your choice, but understand the costs, risks and benefits of your approach before a single line of code is written.

a short intro to neural networks

May 9, 2009 by Sherwin James | 4 Comments

As the new Terminator movie is about to premiere, it reminds me of the key real technology on which the fictional robots operated. I am speaking of neural networks.

As with most things in my life, I am not an expert on neural networks, but I know a little about it.

Artificial neural networks are computer based (hardware and software) simulations of biological neural networks, as those that exist in the human brain. As with the biological ones, artificial neural nets have their “magic” in their neurons and in the interconnections between the neurons. A neural net is trained on a controlled set of inputs and expected outputs, by applying different weights to the interconnects and possibly “tweaking” the formulas and thresholds for the outputs. Once training reaches a satisfactory level of success rate, the neural net is ready to do it’s magic. This “magic” allows for the interpretation and solving of inexact problems.

Several years ago, I had the good fortune of working with some top researchers in the field of artificial neural networks. I learned a lot from them including how to construct and operate a neural net (input, nodes, back-propagation formulas, weights and training). But more importantly, I learned about some of the practical (real-world) applications for neural nets. Those include speech and face recognition, solving complex logistics algorithms, intelligent agents, and building fully automated robots that take over the world.

A good starting point for more info is this Wikipedia page wikipedia.org/wiki/Neural_network.