So Why Should I Care?

The simplest way to put this is:

1)
Within a few years you are going to be programming in someone else's framework in a very limited capacity (frameworks are too expensive for one corporation to maintain properly)

2)
You are going to be helping write a framework but it won't be on behalf of any company - it will be for ALL companies

Software Adoption Options Businesses Now Have

Businesses traditionally have to make a choice when they bring technology in-house to solve a business problem:

They can build from scratch to solve their problem
-
This provides the MOST flexibility for future needs at the MOST cost now

They can buy an off the shelf product
-
This provides the LEAST flexibility for future needs at the LOWEST cost now

They can buy an off the shelf product and customize it
-
This provides the best of both worlds PROVIDED that the software is extensible where it is needed and costs for adoption and customization are low


That's how it used to be. Yet there are two OTHER fairly new options now:

-
They can apply an Open Source Framework and get a substantial level of flexibility at almost NO cost now

-
They can create a picture, via MDA, and GENERATE the EXACT software they need for minimal cost and use that model that they create to MATCH their software DIRECTLY to their business process (so that their class models are directly understandable by the business, usually thought to be blasphemy, and not just the Software Engineers).

There is a significant time investment here but the focus of the Software Engineer changes to one of understanding the business model vs. dictating what the company they work for can and cannot do with the software they create "because it's too inflexible" - no, the engineer who wrote it just didn't understand the business process fully (and probably enough stakeholders were not involved - which is common - the whole "just go build it and we can complain about what you gave us - but we weren't really involved in the first place routine").


I single these two out as different because THEY CAN BE USED TOGETHER.

The Open Source Framework, such as the Castle Project or the Transparent Framework, or both, used in conjunction with one another, are at the PSM (Project Specific Modeling) level of work. This can evolve or change independent of the Business Model because frameworks in general provide a level of "services" to any application that use them.

Oftentimes you need to know how to use this framework and invest the time to do so but you DO NOT necessarily need to understand them fully. This is a key idea I will address later in this post, but let me just say this: the level of knowledge about software engineering you need to have to write a framework is a lot higher than the level of knowledge you SHOULD need to have to use it (i.e. it should be do X, Y an Z and watch what happens). Anyway, here's what we know about software in general:

1. There is a Data Layer
2. There is a Business Layer
3. There is a Service Layer
4. There is a Personalization/Security Layer
5. There is a Presentation Layer

A good framework lets a business focus on JUST the Business Layer - i.e. how to get their job done at the lowest possible cost per unit (to maximize the amount of money going to the shareholders as we discussed previously).

A good framework also lets you extend the OTHER 4 layers via applying simple patterns of code (and handles all the wiring for you). So for example, the Monorail sub-project in the Castle Project framework uses an MVC architecture that allows you to CREATE new pages just by creating a new class, inheriting from a base class and then overriding a some kind of rendering method (you still need to populate the controls on the page yourself - which Transparent when used in conjunction, abstracts for you).

Finally, a good framework DOES NOT make assumptions about how you are going to use it - it tells you what you can and cannot do and hopefully allows you to pick and choose what parts of it you do and do not want. It also is constantly evolving based on user input (which is why communities are a better place for these kinds of things - as they involve frequent release cycles and almost constant feedback which is very difficult for even a software company to manage).

Now, if the framework you use is constantly evolving to provide new features to you, the business model is just the opposite. It evolves as you understand and redesign your core business classes around your new understanding. Although it does need to be frequently evaluated for internal consistency, once it is formed it CAN be relatively stable for a time.

Unfortunately the thing that throws a monkey wrench into this perfect picture is programming languages and standards. As these change often you have to take your whole application and re-write it in some other language or you get "application incompatibilities". But what you CAN do in this situation is to just regenerate your business model in the target language and go and find a compatible framework. This is the concept underlying POCO's (Plain Old CLR Objects) as opposed to building RICH business objects. If you want RICH then wrap the POCO - but I am getting into design here and I do not want to do that. The point is that this whole ugly "increase shareholder value legal obligation of Corporations is actually your friend" - IF you design your applications from this perspective I am advocating here because it FORCES you to design well - the only caveat is that the corporation MUST measure profitability long term and not just short term - and that ALMOST goes against the increase shareholder value mandate

...however, it is your worst enemy if you do not design your applications from the perspective I am advocating.

So What Kind Of Developer Are You?

If you are "from scratch" developer and start with a VS.NET blank slate every time you go to code - like you are ready to free your mind, be creative, and draw on a fresh sheet of paper, this next section is for you. Do you like Design Patterns - the Gang of Four, often talk about architectural trade-offs of applying design patterns, attend user-groups, favor Eric Evans Domain Driven Design (actually get excited about this stuff)? Are you really interested in quality and applying true proven software engineering practices to your work? Do you model new architectures in your sleep and grab a notebook and scribble how you can apply a decorator with an MVC controller and an Extender to get an extensible UI framework? Are you one of those people that cares about the code you write? Do you look around you and just say "I can apply a design pattern here and clean up all this bad code"? Do you also say why won't THEY give me the time to do a "black-ops" project, under the radar of the project planning folks in a "refactoring window" with some of my peers and get us out of this sea of bad code? Don't they care that it takes us 5 times as long to get anything done because everything we do breaks brittle code?

You my friend, are like me. You really want to "improve" the environment that you are in. That's a BIG problem. Shame on you :) Unfortunately, that kind of thinking does not USUALLY belong in a corporation - they can't afford you AND make a profit - so you MAY want to re-evaluate why you became a software engineer in the first place. Or let me put this another way, they can't afford you and make a SHORT TERM profit for their shareholders.

Why, you ask? Well, I had to learn this one the hard way:
There is a substantial cost to ideas

This is where the worlds of the Pragmatist (who is about being effective) and the Idealist (who is about seeing a vision) collide. Actually, we have BOTH aspects in each of us but corporations encourage the pragmatist (even the blind pragmatist) but spend lots of money to get the idealists only to put them into a place full of pragmatists and purely pragmatic - risk avoidant thinking (or process heavy environments, in an attempt to manage risk) - but that is a different blog (it's also one of the practices that go against the corporate legal mandate because they are frivolously wasting shareholders money and should be taken to task for it - how much did YOU with all your idealistic vision cost? and here they are WASTING resources).

Consider that you may be in design pattern oriented thinking and your manager last programmed using COBOL. First of all, they are not going to trust you - so you have to build up a lot of "respect capital" to even begin to float ideas as earth shaking as design patterns. You then have to prove you can dig a ditch their way (yes, I know they didn't hire you to dig a F#$%#T#@ ditch - just dig the ditch). So by now, 3-5 years have passed (if you dug the ditch), you have proven yourself on some projects, and you get your opportunity only to be told that your job is being outsourced to India (and they should - you wasted their money by NOT sharing your ideas or sharing them - I get so confused about this point). So you never get around to that design pattern that was going to revolutionize their environment (or worse you actually believe that you CANNOT do that - when you most certainly CAN). Oh, guess what? You get to start all over again - lucky you :)

What you may not realize is that since ideas cost time and money for people to understand them - and if you are a poor communicator you are really screwed - you end up fighting a POLITICAL battle when you are NOT a political animal. This has a LOT of implications. For example you have to "champion ideas" so you better be a really good orator - that's why you decided to lock yourself in a dark room with a computer right? You were good at public speaking? Masterful at the art of persuasion? Not likely. You are just damn good at what you do. However, BECAUSE you made the attempt to float an idea (I pity those that never at least try to float an idea), because you did that, you actually, unbeknown to you, have invoked the wrath of the increase shareholder value legal mandate of the corporation.

YOU are costing the company money for having an idea, taking up people's time on the clock with your idea (so you should share ideas only in your off time :)) and trying to make a difference and BE a stakeholder in that environment (in other words, care)!


So there is a substantial deterrent in corporations that do not rely on Agile processes, where risk is minimized by shorter release cycles instead of processes designed to manage risk by taking away your initiative.

ALL corporations, unless they follow agile processes, MDA OR have software as part of their core competency - and everyone right up to the president of the corporation wholeheartedly embraces that as a key differentiation strategy of the corporation, have this kind of environment.

So most likely, you are working for a company who has just added your "pursuit of what is right" to the cost of selling their widget and if you AND your team can't justify WHY what you are doing will benefit the company - have fun. I personally did not spend the last 10 years of my life doing software engineering because I really wanted to go into politics. In fact, MOST software engineers I know HATE politics - but are of course, very opinionated about it.

To my credit, I now have learned the hard way what not to do and hopefully you will take away some of that knowledge by reading this blog. I now KNOW how to work in Government and the Private Sector and with companies that are in COBOL - Procedure Oriented mindsets and so on. They were some hard lessons learned, let me tell you.

That substantial deterrent also impacts your family, because it affects your financial well being - so you better NOT have ideas - or share them unless you can point to a graph and say:

"see here - this is where by doing this design pattern we shave off X amount of loading time and that translates into .50 cents per transaction and therefore with 10,000 transactions a day this improvement saves $5,000 a day!"

That is exhausting - to make change you have to be a business analyst, orator, programmer and a salesperson. Lucky you.

Any good businessperson worth their salt is going to have you justify to them how what you are doing affects the bottom line. Yet an alternative way to handle this is to do the change and SHOW THEM the results - but for that you need smaller release cycles (and you really need those smaller release cycles because THEY ONLY THINK THEY KNOW WHAT THEY WANT). If your stakeholders only saw that it made a 30 cent improvement and the project was for a 50 cent improvement per transaction then rewrite it - only agile organizations operate this way however. In most organizations, you need to justify it, it has to get approved by 5 people and then you can apply it to a DEVELOPMENT environment - that's the whole CMM Level 3 - whole lot of disconnected waterfall documentation - type stuff - it's also the reason the Dot Coms, in the Dot Com Boom, demolished the big 5 Consulting firms because by the time that the Big 5 Consulting firms got the approval for the 5th person, the market had moved. It's also why the Linux Open Source types have rocketed past the existing software development companies who sat on their laurels for a while thinking that their market dominance could not be challenged.

So bottom line, having ideas exposes you to risk - If you are someone with a lot of ideas then you need to know how to channel them and Corporations are NOT the place to find out how to do that.

The only exceptions I have seen to this are organizations that:
1) Embrace Agile Development
(shorter iterations allow you to "try" things)

2) Embrace MDA
(because the business is looking at you as MORE THAN just a engineer - your modeling and understanding of what dumb computers need to know to do things leads you to be precise and this is necessary in assisting at defining a model)

3) Derive their core competency from the work you do

Now, #3 is a little tricky and I really don't trust it. Because it is NOT ENOUGH to have the organization JUST derive their core competency THEY HAVE TO UNDERSTAND THAT THEY DO. I had one company I worked for that hired on a new financial person and a new director of IT, who did not understand that the SOA, BizTalk, aggregation architecture we had implemented WAS DIRECTLY TIED TO our core competency and therefore sought to outsource it (of course :) - I now know why my architectural mentor at that company became so cynical - he took things too personally - so did I at the time). Yes, it was retarded to do so, as they saw things as "too complex". Granted, there was some unnecessary complexity there but we were not dealing with Microsoft Access here either - this was a mid-size company that was RUNNING off of this architecture and that had phenomenal growth from a start-up, integrating all the "stove pipe" systems and the people who ran those systems, but who never actually talked to each other (yet dysfunction manifests in IT), BECAUSE of it. All we needed to do was reduce some of the complexity and not throw the baby out with the bathwater. So what I am trying to say is #3 implies a lot of things and if you do end up working for a company in that category make sure that the commitment to "a world class software team" is IN the company mission statement - and if the IT Director leaves - get ready to leave too - IF you really care about this stuff that is.


The short answer here as to why you should care because everything is moving towards the Framework based approach where you just make SMALL code changes to a pre-existing infrastructure that somebody else needs to maintain - so they don't need you full time in a programming capacity.

Even SharePoint, a product, is actually 1) a product 2) a framework and 3) a platform. So this is not really in the category of buy and customize (such as Oracle is), because it is all inclusive. You need to really determine what kind of developer you are and very quickly plug into that level.

If you are content just working through existing extension points, then get as much of that work while it is still available - you should not need a lot of software engineering experience to do that (but also that means your job can be easily outsourced and before long some smart person in Open Source will write a program to write your code using reflection).

However if you want to actually WRITE quality software we are fast approaching a time where that WILL NOT be able to be done in all but a few corporations (the corporations that have the vision to see its benefits, can wrap their business processes around customized software, adopt agile, adopt MDA and take software engineering seriously) - everyone else should just go get SharePoint - (or Drupal for the Open Source community), and leave the real software engineering to the people who want to do it.

As always, I look forward to your feedback (and the links are on the right).

The Alpha Release Download is Available!

I finally got all the project files for the Transparent Framework uploaded to sourceforge.net and have prepared my first formal release of the Transparent Framework.

When downloading the Transparent Framework be sure to take a look at the release notes icon for instructions on how to get started.

So you can go get it now :)


To summarize my last post, what is happening in the software industry as a whole is that software is beginning to write itself. The tools are becoming more and more sophisticated that "programmers" are no longer needed to any level of degree than they once were. Combine that with the massive levels of outsourcing and you can see pretty quickly that your value as a "programmer" is limited at best - someone else can already do it cheaper than you can AND pretty soon machines will be doing it (I give it less than 10 years at the most).

So where do you fit in then - if cheaper is the yardstick of measure as corporations have a LEGAL mandate to make profits for their shareholders (they don't have a LEGAL mandate to do anything else)?

A wise person I once worked with is a QA genius, he once told me that our salaries are overly inflated if we are just providing a "tool" for people to get a job done and now you need to be a "multi-use tool".

By understanding that:

1. The LEGAL mandate for a corporation is to make profits for shareholders - click on The Corporation link on the right side of the page.

2.
The World is Flat - click on the Charlie Rose interview with Thomas Friedman on the right side of the page to view this link.

3. Corporations, based on #2, MUST find the lowest cost.

4.
The lowest cost is MDA, Framework based code OR a platform that does everything disguised as a product (because many corporations cannot even afford to pay people to write quality code anymore - and STILL make profits for Shareholders - when was the last time you heard that we have X deadline and therefore quality is the first thing to go right out the window?)

5.
We are right now in a pre-MDA time (i.e. code is not writing itself yet but Business Analysts can draw pretty UML pictures and get code generated for them)

...you are now in a position to do something about it.


In actuality this is a tremendous opportunity for you, but you have to see it. The cheese has moved. Open Source Frameworks and Open Source Solutions are increasing - not decreasing. We recently saw a whole language, Java, get released to the Open Source community. Ruby and the Scaffolding concept quite literally came out of nowhere, RSS came FROM bloggers and pushed up into businesses.

Personally, I got into Microsoft.Net because it was going to be the LAST language I thought I would need to learn. I wanted to invest heavily up front and spend time reaping the rewards spending quality time with my friends and family. That's why I've specialized in it since 2001. That however, was a naive assumption. I did not understand the big economic picture. You cannot stop the flow of change and you need to know where change is coming from - and it isn't technology.

Technology does not change because one technology is better than another. I made my assumption based on the fact that .Net was inclusive of so much and with generics and delegates it became inclusive of so much more - but that is naive.

Technology changes because corporations can build a widget for 10 cents less then they could a year ago and there is a flurry of activity for everyone to learn that new "thing" that lets them build the widgets cheaper than their competitor. That's how jobs are created - flurrying from one technology to the next - or at least that's how jobs WERE created.

That model does not account for two things:

1. Open Source
- a lot of people freely contributing to the creation of software just for the recognition of their peers (which if done for a corporation, that software creation recognition is non-transferable) and the desire to use QUALITY as a measure and NOT cost per unit - which undoubtedly produces higher quality software (because a corporation generally CANNOT produce high quality software, documentation, support and the like and still MAKE PROFITS FOR ITS SHAREHOLDERS - corners need to be cut). As a side note, if you try to hold quality as a measure then you will probably be fired UNLESS you work for a software company or some company that has a core competency where QUALITY actually lowers the unit cost (or raises the number of units sold somehow).

2. The evolution of our tools
- As I said before, our tools have evolved so far that they are now preparing to make specialization itself obsolete. You can't make decent money as just a programmer (look at the web sites that specialize in "hired gun" coders who are building whole systems for under $100 US - that's a lot of money in some countries), - you need a lot more tricks in you bag (i.e. DB Architect, Software Architect, Business Analyst, QA Analyst, Administrator, Modeler, etc.). Cephas Consulting even goes so far to say that you need new titles (to adopt MDA), like "DB/Software Architect Business Analyst Modeler". SharePoint is a tool so is the TransparentFramework - I am not saying they are anywhere near on the same level, what I am saying is that ANY TOOL if it lowers unit cost MUST be used - by the way, YOU are part of unit cost.

As our tools evolve and more Open Source projects become available AND the standards bodies such as the W3C, OASIS, etc. - figure out how to make everything more inter operable - we get closer every day to MDA and that is the true goal.

So IBM, Microsoft, Oracle, Sun and Novell and just about any other technology corporation out there are just blips on the radar, who in their moves to seek profits for THEIR shareholders in the short term, actually fragment the simplicity of us getting to that MDA space (but this is not uncommon in this country as infrastructure investments need to be paid for and that is why the 3rd world is moving into solar power instead of building power plants - and why we just recently started rolling out fiber networks here in the US when they have been available for some time in other countries).

...and we are subject to learn as fast as we can whatever competing technologies that they these companies create. What happened prior to .Net was that Microsoft was holding on to its core VB developers and trying like mad to maintain backwards compatibility (so THEIR customers shareholders could make a profit by not needing to re-architect their environments that worked fine the way they were currently designed every year). What they rudely awoke to was that the Open Source Linux community adopted Java and the Java community was where the bulk of the people lived who were pursuing Design Patterns in pursuit of MDA, and in doing so the quality of their software shot up to engineering standards AND the Open Source community for FREE started creating software that was in some cases a lot better than a corporation like Microsoft could produce AND STILL MAKE PROFITS FOR ITS SHAREHOLDERS.

Once Microsoft's competitors figured this out they heavily funded Open Source - AS A BUSINESS STRATEGY FOR THEIR SHAREHOLDERS - not because they really cared about producing QUALITY software. It was a business decision. As a side note, in my exposure to Microsoft as a Level 3 Support Engineer, I experienced the best job environment I have ever been in - Microsoft really does an outstanding job at taking care of its people and its contractors - but I fear that the LEGAL mandate of making profits for its shareholders, as times change, will transform that organization from the internally open culture it is to one that is more and more concerned about the bottom line and the metrics based approach - less focused on collaboration and more on competition - even though the ideas in The Cathedral and the Bazaar - the philosophy of the Open Source movement - indicate that real productivity comes from doing what they have been doing (I am just not sure how to make this work IN a corporation). It's a different world and it's going to get stranger and more unrecognizable.

So where do YOU fit into all of this?

Well, right now the place to be is in the Open Source community creating or helping create frameworks or projects that will lower the unit cost of systems built using those frameworks - so you and your work has some longevity outside a single corporation AND you can produce high quality software if that is what you want to do (corporations are not the place for this - you want to stick to realities there, what is needed NOW and make sure that you adopt, not create unless you have the backing to do so, an infrastructure of some kind to handle capabilities - providing extension points for what you can do LATER so you don't get pinned in and don't end up swimming in a sea of badly written code so you can hit your deadlines).

A Framework in a lot of circles is no longer thought of as a "competitive advantage" - it's a necessity that far too many organizations don't pay attention to and this hurts them on the extensibility, debugging, company image and maintenance front once their product releases either internally or externally.

Alternatively, right now the place to be is into using tools - or writing tools like SharePoint, where you have a product but that product HAS a sophisticated framework that allows you to use that product as a PLATFORM so you can lower your unit costs to build an application by LEVERAGING as much as you can out of that Product/Platform that DOES all of the things you need to do AND provides room for capabilities and extension points. Unfortunately some of the licensing models do not support this approach unless you are a large corporation and can afford it.

There are a LOT of completely open non-proprietary technologies out there such as JSON (object oriented JavaScript), XFORMS, XQUERY, XPATH, XSL, XML, etc. - but even using these technologies is not really a hedge against needing to apply a business process to yet another technology and that hedge for you is MDA (unless you like re-inventing the wheel every year).

In MDA you have a CIM (Computer Independent Model), PIM (Platform Independent Model) and a PSM (Platform Specific Model) - and this is THE way to build your applications as the PSM (where the WCF, WF, WPF, InfoPath, XAML etc. lives), is actually generated from the PIM. It does not matter if you want Java, Ruby, C#, PHP, C++, DDL for Sql Server, DDL for Oracle, etc. - So on one hand, you have community frameworks emerging and on the other, you have Products that "can do everything" that are really Platforms in disguise (and the best way to hook into a technology stack like .NET 3.5).

MDA offers a different approach as each model is used to build the subsequent one (CIM->PIM-CSM), and the "frameworks" are just plugged in at the PSM level - the final product being not a platform disguised as a product, but a single product to serve a very specific business need that that can be regenerated as needed from the model. If the business process changes, make the change to the model and regenerate the product. The latter approach is to "just build a platform disguised as a platform" or use a "framework" to account for 90% of your scenarios - it is more inefficient but it is where we are today as MDA has yet to mature - but keep an eye on it.

I think the most important thing here is that you start looking for new cheese (read - Who Moved My Cheese), preparing to deal with the changes ahead and a good start for you is downloading the Transparent Framework to tweak your thinking a bit from a technical perspective on what a general purpose framework should provide. You are going to want to have a number of these "frameworks" in your back pocket and you might even want to get some public recognition by contributing to it :)

Overcoming Resistance To Change

I wanted to tackle this topic first, before getting technical about the software engineering framework I wrote, as I feel that software engineers are constantly walking the line between producing quality and getting the job in front of them done and this is why I wrote the framework.

The irony here is that within the next 5 years, the battle over producing quality and getting the job done quickly for the lowest possible cost is about to end only to be replaced with a MUCH harder challenge ahead.

The next challenge ahead is how to leverage the work other people have created, and to do that you are going to need to understand how to deal with resistance to change. Right now, changes are taking place that are so disruptive that they are making specialists in general, a dying breed. This is time of tremendous opportunity but also tremendous upheaval and it is easy to lay blame all over the place. The problem, as you will see later in this post, is not with more, better, faster technologies but rather in the thinking of the people who use those technologies and how to implement those technologies with a minimum level of friction.

Part I: Understanding The Big Picture

Put succinctly: We have a picture emerging where we will have no training for a barrage of technologies hitting us that we are expected to fully understand and implement because it provides the lowest cost per unit. If we do not do so, our job will be outsourced to a different country because the only legal responsibility of corporations is to turn a profit for their shareholders AND we actually have been encouraged to believe that their "job" is to take care of us.

I had the opportunity this year to work for Microsoft as a contractor on the SharePoint Development Support Team, so when you called Microsoft for support you may have spoken to me. This was a very eye opening experience for me and one that left me with a very different impression of what "a level 3 support technician" was - but that is a different story.

More than anything, after being exposed to this technology, I left with the impression that even Microsoft is starting to see that in the next 5 years, very few of us will be writing code and they are leading a charge with SharePoint 2007, that completely blurs the line between Administrator, Database Developer, Web Developer, Security Admin and Business Analyst. Your role in the traditional sense, is gone - and quite frankly, Microsoft is somewhat behind the curve.

Einstein once said, that "our technology has exceeded our humanity".

We are fast approaching a point in human history where our humanity better catch up with our technology real fast. The challenges that lie ahead are what I call "awareness challenges" and will very much determine how easily or how difficult it will be to adopt new technologies.

Microsoft this last year barraged the marketplace with a dizzying array of new technologies - everything from an new scripting infrastructure (PowerShell) to a completely new framework of .Net technologies (WF, WCF, WPF) to SharePoint (and expect .Net 3.5 this coming year - and don't even get me started on Windows Server 2008 and Vista).

At the same time, we had massive levels of outsourcing taking place to China and India due to the billions of dollars of Fiber Optic Cable laid down during the Dot Com boom that allowed businesses to lower their unit costs by digitizing and farming processes out across the planet to seek the lowest possible cost and therefore cut back on training here in the US. So now you are competing for work with every other man woman and child on this planet (and you probably don't know that) AND no one is training you how to use these new technologies (so they aren't getting adopted).

Meanwhile our attitudes of entitlement are being progressively encouraged to create our own perfect little "entertainment bubble" where actions in that bubble do not really affect the physical world we live in one bit and we do not develop the humanity we need to work with one another. Our awareness is actually being dumbed down so that short term profits can be made and there is a helluva storm on the horizon.

In addition, we saw the continued rise of the open source moment and the creative commons movement, (or the commons movement), as direct challenge to the the existing ownership structures of the corporation whose only functional directive is to make profits for shareholders (so by definition they must seek the lowest cost - in any country). By the way, corporations cannot not take care of you because they will be farming out your job to the lowest bidder in a different country so the model of entitlement you may have.... Well, you had better re-examine that pretty fast.

So the convergence of these factors (new technologies, our attitude of entitlement and corporate legal responsibility to shareholders to turn a profit at the lowest possible cost), is leading to one place:

It is the holy grail of software engineering - draw a picture and get the computer to write the code for you to create a fully realized product - and it is what will produce software at the lowest possible cost. The only thing is, you don't get to write the code anymore.

Part II: Disruptive Technologies

By now you are probably thinking - what does this have to do with Software Engineering?

Let me explain. Microsoft is finally getting it. That is why they barraged us with technology this year - they are playing catch-up. They stopped trying to "coddle" their customers by insulating them from really disruptive changes because they see the 800 pound gorilla that is coming.

MDA (Model Driven Architecture) is coming and if you think that India with all of it's people having PH.d's. is a challenge via outsourcing, you have not seen anything yet. Today, I can draw a UML picture in Sparx Enterprise Architect and it will create my database tables, C# code, XML, Java code, PHP code, etc. Then I can take my code, import it into the tool and it will create my UML Sequence Diagrams! Add a plug-in such as Tangible Architect, and I can then create a full application.

Frameworks created by communities are coming and are the wave of the future - plugged of course, into MDA. I am particularly fond of the Castle Project (in fact, the Transparent Framework is designed to use ActiveRecord as it's MVC model - although it can be adpated to use other model types). They have a "scaffolding tool" for Microsoft.Net similiar to the one used in Ruby, that can look at database tables and produce a full MVC architecture that you can tweak by just changing the physical files - you need something else - go ahead and create a new scaffold - keeping some files and dumping others.

MDA concepts (i.e. you write less of the "plumbing" code to make your applications work and plug-into existing design pattern based, infrastructure), are giving rise to Global Web Based Collaboration Platforms. SharePoint is Microsoft's answer to a Global Web Collaboration Platform, and a good one at that, which you can do just about anything with (if you know how and you NEED to know how). It gives you three key things - 1) Extremely fine grained structures for permissions, 2) Web Parts - so your business people can build what they need like Lego blocks (and you don't have to think about realities and can focus on capabilities) and 3) event triggers when items are added to lists - so you can almost completely decouple your systems.

The 800 pound technology gorilla is MDA - but that is even tiny compared to the challenges ahead of you and I. The biggest challenge we face is adopting these new processes and technologies and changing our thinking and that of our customers and peers to be ABLE to adopt them BEFORE MDA arrives. So what we have is a pre-MDA world from a technology perspective and a whole bunch of technologies are going to hit you before MDA comes along that you are going to be expected to implement at the lowest cost. Because your job is not "entitled" to you - the lowest cost means outsourcing in India, China or Africa too. Most importantly, lowest cost is a certainty - as the ONLY legal responsibility a corporation has is to make a profit. In other words, we are going towards MDA (and the semantic web), whether you like it or not - the only question is will you be a casualty of it or will you thrive because of it.

You new job will be trying to plug-in to as many of these technologies and deal with resistance people are going to have to change that these technologies present BEFORE we get to MDA (at which point if they have jobs in the same area, they will be radically different then they are today). Their lives WILL be affected by these changes in the next few years (just like how the ASP script web developers got wiped out when .Net came along - but the fallout will be a lot more massive). The technology part is easy. You have to get them to understand that the world that they live in tomorrow may be one that they are not psychologically prepared for.

Part III: Preparing People For The Changes Ahead

I realized today that a persons level of awareness is directly connected to what they can and cannot accept.

You can think of it as a "box" around that person which forms a complex system that implicitly constrains their actions. One level of awareness is not superior to another - they each however offer different perspectives and result in different actions. A simple example of this would be to answer the question would you rather be in a dark room with a lion or in a lit room with a lion? I am not so sure that being able to see the lion before it eats me is what I want - but then again, I might have a fighting chance at surviving so its hard to say. Thus each level of awareness has its own positive and negative aspects.

So, if you want someone to do X then create a culture of voluntary action by understanding what level of awareness is necessary to "allow" X to occur. Applying a rule (or an overbearing manager), will most likely not produce the outcome you want.

Take for instance, introducing SharePoint into an organization. If you are going to do that you need to recognize that people who are specialists or good at just one thing, are going to have a problem because SharePoint largely erases those boundaries between groups and encourages people to stretch themselves in ways they probably do not even understand (i.e. developers need to do admin tasks, security zelots need to at least lighten the reins in development environments, you may NEED to create a set of environments, etc.).

These people may create a great deal of resistance to its adoption but what THEY need to understand is that the world that they live in is very different today then what they think and then you can point them to Thomas Friedman's The World Is Flat - to show them that their "model" of the world is NOT accurate and cause them to re-examine their perceptions and thus lower their resistance to SharePoint (because now SharePoint should be the least of their worries), or help them deal with their fears as people often oppose things that are outside of their level of awareness.

So as a simple example, consider walking across the street:

Awareness ->I can walk out into the street and I will be safe
Action -> Walk out into the street
Effect -> You get hit by a car and die

New Awareness -> If I walk out into the street I MAY not be safe (so I cannot blindly do it)
Action -> Look both ways BEFORE crossing the street
Effect -> You successfully cross the street


In short, your level of awareness allows you to navigate the world. If your "model" of the world is only partially inclusive of challenges you face, then you are at the mercy of those challenges but to be fair, you also benefit from not needing to concern yourself with those things (i.e. the lion).

People in general do not often have a need to change their level of awareness unless an answer to a problem they have is outside of their current level of awareness (and if you try to force a new level of awareness on someone who has not acheived it, then you will just be applying an endless series of rules, "shoulds" and protocols). Their lives/routine for instance, may look like this: wake up, go to work, watch tv, go to sleep, rinse, repeat.

It's important to note though that information is not enough - a true change in awareness implies a change in meaning and value for that person. A change in awareness may change that basic pattern I mentioned if for example, that pattern is based on entitlement. So after the change, the person may include in that pattern of actions mentioned - update my skills - and in that moment, though humanity and not technology, you have successfully overcome one of your biggest obstacles.

Having the skill to successfully overcome resistance by encouraging a "voluntary system of action" such as what was acheived in the Cathedral and the Bazaar - the philosophy of the Open Source movement - will be much more important than ANY technology you learn.

I hope that this post sets the stage for many good conversations with you and I look forward to hearing your feedback. If you want to refer to any sources mentioned in this post check out the References section in this blog.