The free exchange of ideas has been the principal driving force behind innovation in every field of human endeavour. Our culture, politics, laws, scientific knowledge, technology and the development of the human condition have all been enhanced by collaboration and the flow of ideas through the exchange of information. In our current epoch the technologies that enable the open exchange of information are available at the finger tips of a third of the worlds population yet, powerful cultural, legal, territorial and commercial disincentives to the free exchange of information remain, and in many areas of science and broader society, a culture that restricts collaboration prevails. Beyond these threats to the transparent and free exchange of information is the growing entropy that results from propaganda and ignorance which continues to erode the value of information and increases the cost of its extraction.

An elegant and proven solution to these problems is the creation of a open regulated exchange, bringing together producers and consumers of information and facilitating secure and transparent exchange. Financial exchanges, if they are governed properly, provide a fair platform for participants to exchange their goods and services and or engage in speculation. If they operate efficiently and provide services that are demanded by their customers the exchange increases the volume of trade to the financial benefit of all participants. Information exchanges can do the same for innovation.

My thesis is simply that the creation of Information Exchange which I define as a platform of services and a system of governance that facilitates the exchange of information between participants that create and/or consume information will result in greater collaboration between the participants and the increased volume of information exchanged will drive innovation. What is more I propose that such exchanges can be created for any sector, whether it’s government, health care, military or scientific endeavour, an Information Exchange will drive innovation, making its application universal. Finally by implementing financial speculative instruments based on the value of information as measured by its utility both within the exchange and externally, the innovation will be self financing with no need of state funding.

Conceptually it is useful to consider the Information Exchange (IEX) from two different perspectives, organisational and technical. From an organisational perspective the Information Exchange is an institution which is dedicated to catalysing collaboration between its participants and protecting the information assets they invest in the exchange. To this end the IEX seeks to shoulder the burden of legal, regulatory and security compliance on behalf of its participants. Furthermore the IEX endeavours to reward its participants not only by ensuring that any innovation that is attributed to their information asset is fully acknowledged, but also by offering access to unprecedented analytic services for the mining of the rich seams of data which the exchange aggregates from its participants thus giving all participants a foundation for innovation.

From a technical perspective the IEX is a secure computational hub, an enterprise system for the analysis of digital data. The system facilitates the rapid and secure exchange of information resulting from analysis carried out on an aggregate of public and private data; information assets invested in the exchange by participating members.

The differentiation between information exchange and data sharing is an essential proposition of the exchange. Data sharing requires the data owner to agree to provide access (and potentially a wide range of other rights as well as obligations) to data with other parties under a contractual agreement. The sharing of data will typically result in the data being duplicated and distributed to multiple consumers. Obtaining data sharing agreements for privately held data or data of a sensitive nature is typically difficult, time consuming and expensive. For the data producer the problem is compounded by multiplicity of agreements which they may be requested to negotiate and for the consumer simply identifying relevant data for their analysis can be thwart with difficulty.

Information sharing within the exchange is the sharing of the results of an analytic service provided by the exchange between participant members. While the underlying data used in the analysis may be private and sensitive in nature the results of exchange services based on the data will be cleansed of information that could be used to compromise the confidentiality of the data and will ensure compliance in legality, regulation and security. Data providers need only negotiate terms of sharing with the exchange. This paradigm provides the benefits of sharing data by allowing participants access to information from analysis conducted on that data but minimises the risks and costs associated with sharing data with multiple consumers. The data is never shared, rather the information is exchanged.

In my next entry I will describe in more detail the mechanisms available for participants to invest information assets into the exchange and the underlying IT architecture of an exchange.

I’ll start this entry by briefly recapping the two metrics I used to measure the performance of our old build process discussed in my last entry:

• Build time. If it takes you longer than a few minutes to build your domain, you should be asking why?
• Work time. If it takes you longer than a few minutes to make a configuration change to your domain, you should be asking why?

I have yet to hear a valid reason why a standard WLS domain (e.g. not a Portal domain) should take longer than a few minutes to build, if you think you’ve got one please post a comment on this entry. The main reason I come across is that the build process is “Dynamic” i.e the domain is created using WLSHELL or WLST. In my experience these domains take between 10 – 40 minutes to build. If you use these tools to build your domains please post a comment on the time it takes you. If you’re happy with this time and it meets your requirements, well thats marvelous. After about the fourth time I rebuilt and deployed a domain of ours to fix multiple configuration issues, a process that took about 90 minutes (after two days spent trying to figure out exactly where to make the changes), It was plain to me that a 20 minute cycle was not acceptable for us. So there you have it, this is why I believe these two metrics are important. As the old cliche mandates time is money.

The third metric I want to discuss falls under the category of complexity, specifically the number of files in your build process that it takes to build your domain. I appreciate that measuring complexity in this way is blunt to say the least but we have start somewhere and in my experience it is a worthwhile measure. Latter on I’ll discuss another form of complexity that is can’t be measured easily but can be avoided. In my original rant I noted that our build process had no less than 69 files to describe a our WLS domain, thats property files and templates. How many files does your build process use?

In an effort to cap the number of files required to build our domain we came up with the following directory structure for Choom (the version on Choom that will be released in codeshare will be even simpler):

(missing image filestructure.png)

Choom stores property files in the properties directory and templates in, erm, the templates directory, no surprises there. In my last entry I described the hierarchical nature of the properties, this hierarchy is observed in the directory structure.

Directly under the properties directory we find common.properties. You guessed it, these properties are common to ALL our domains. Now there aren’t many properties in this file but this does not diminish its importance, you see, the more properties you can define in this file, the more standardised your domans will be and standardised domains mean less maintenance effort.

Under the domains directory we store the properties for the various domains. Each domain type is named, for instance, we can see depicted the mydomain type. Typically the domain type is named after the project it belongs to. Domain level properties are stored under the domain type directory. As before the more poperties we store at this level as opposed to the environment level the more standardised our domains become.

The templates generally follow the same naming and oragnisational conventions. If you read my last entry on this subject you may have noticed that the use nomenclature is a key tool in reducing complexity. By adhering to some simple conventions for naming files, properties and tokens it becomes easy to locate and make changes to configuration values.

When we first embarked on creating our new build process I wanted to make sure that we stuck to just that, creating a process to build our domains rather than an application that builds our domains. This is why I really wanted to use Ant rather than a programming language. I have come across home made build applications written in Java, Lisp, Scheme and of course Jython/WLST and in almost every instance you find a build team that is not only responsible for environment provisioning but also has to support their miserable build application, yet another reinvented square wheel. This is a surefire way to inject a huge amount of complexity into your build process and I maintain it is completely unnecessary for the vast majority of WLS domains. I should qualify this by saying that Jython and WLST have an extremely valuable role in build verification and environment prooving and my next entry will describe how we exploited these features in choom.

Comments

Comments are listed in date ascending order (oldest first) | Post Comment

• An interesting article. I have used some of your suggestions in the building and deploying of my WLS application. I was wondering what you use for release control of application builds from the source control system. I was wondering if you had any thoughts about Maven. All my current build is done using ant command line scripts. I am looking for a tool or framework to formalize the release of a build into testing and production.

  Thanks

  Ramdas

  Posted by: ramdas on March 6, 2007 at 3:45 PM

• Thanks for the feedback Ramdas I hope you found the suggestions useful. We do use a source code control system but in a very simple fashion. I have found building infrastructure components does not require the same level of source control as application source code. For example we typically find there is no need for branching. Curretly we have clearcase setup to automatically update everyones views as soon as one person makes a change and we deliver code to the build users view every night.

  We all build from the trunk and the build always marches forward, every time we finish a new template we label the build and continue moving forward. One day we may require more sophisticated use of CM but to be honest I see no requirement for this.

  With respect to Maven it may be useful for keeping track of and building software products but I don’t see it being of any use in building infrastructure or applications. I may be wrong but if I can live without it now I have to ask what does it have that I am missing? I’m not interested in integration with eclipse

  If you’re interested I have started a project on codeshare called genesis that takes choom to the next level. We are in the planning phase right now and your feedback would be most welcome.

  Posted by: hoos on March 10, 2007 at 4:44 AM

• Hoos,

  Thanks for your thoughts on Maven.
  On a similar topic, i have been debating where to package the 3rd party JARs when creating and deploying the EAR. Currently i am not consistent – some of the JARs are in the /lib folder and some are packaged in the META-INF/lib folder of the corresponding JARs within the EAR file. I would prefer to have all the stuff packaged in the EAR, but then i noticed that the size of the EAR file was growing given the size of some of these 3rd party JARs – especially these days with opensource frameworks and their dependancies.
  What are your thoughts on where to have these JARs?

  Thanks

  Posted by: ramdas on March 12, 2007 at 7:06 PM

• Thats a good question, if your using WLS9+ then consider deploying your jars as seprate libraries. I have not tried this myself due to time constraints but the principle seems sound. Let me know if you do try this out. Otherwise I suggest placing your jars in the APP-INF/lib folder of the EAR and removing them from the orginal EJB/WAR files. Be careful though, make sure you test the app after every change to the packaging.

  When you say your EARs grow in size I’m not sure what the problem is. For instance say you are deploying two EJBs in an ear file and each EJB has it’s own copy of a dependant jar in it’s META-INF/lib directory then placing one copy of this jar in the APP-INF/lib of the EAR and removing the ones in the EJB should result in an overall decrease in size. Please expain if I have misunderstood you.

  Posted by: hoos on March 12, 2007 at 7:21 PM

• Hoos, I am using WLS9.2 and i have read about the concept of deployable libraries.. will go back to it and see if it fits into my framework. When i said, the size of the EAR is big, i was not referring to 3rd party JARS being duplicated between the various modules within the EAR (viz. WS, UI, EJB) but the fact that if these 3rd party JARS could be removed all together from the EAR and be made available in a place where the WLS servers could load them during startup (viz. /lib). One obvious downside of this is that when deploying such JARS to a cluster, i would have to make sure these 3rd party JARS are sync’d up to all the managed servers in the cluster. If they were all part of a single EAR, I would be limiting the number of files to be deployed to the various managed servers to a smaller number (ideally 1) which would be simpler to administer and manage.
  Any thoughts?

  Thanks

  Posted by: ramdas on March 13, 2007 at 12:33 PM

• Ramdas, limiting the number of deployments to clusters is a good idea, and it’s good that your thinking about it. I currently have to deal with a cluster onto which we deploy over 20 EARs, I consider this to be plain wrong. It is a perfet example of a lack of appreciation for operational requirements.

  Ensuring the correct version of each application is installed on every machine in the cultser is something our build, installation and deployment processes are responsible for e.g. we use ssh to execute scripts that check the versions of the binaries on all machines before deployment commences.

  On production servers we enable cluster constraints which fails deployments of applications with different deltas.

  Have a read of the various deployment options available, I prefer nostage and external deployment options to staging as I have found staging took a long time, but your situation may be different.

  Posted by: hoos on March 15, 2007 at 3:45 PM

• Hoos, where would you package 3rd party JARS like JSF and other open-source frameworks which are used by the modules within the EAR – these are not necessarily shared between the various modules in the EAR. Currently i have these JARs in many different places(EAR/APP-INF/lib, EAR/WAR/WEB-INF/lib ….). BEA recommends in some places to place 3rd party JARS in the APP-INF/lib folder which is at the EAR level and then at other places in the document i see it recommending to be added to the WEB-INF/lib folder at the web JAR level – and then in some other parts of the doc it says to place these completely outside the EAR but in the Domain system classpath via the /lib folder… i am confused…

  Posted by: ramdas on March 16, 2007 at 7:31 PM

• I wish I had listened to this advice a little more about using ant to build the domain rather than WLST.

  I’ve taken us down the WLST path of configuring domain templates and unpacking them to the servers. For the most part it works rather well. Unfortunately there are missing pieces – for instance you can’t pre-configure logs to ‘rotate on startup’ or to limit the number of log files. Oh you may set those values in your template, but they are lost when you unpack or write the domain. I’m also running into similar problems with the assignment of some resources to a target or cluster when configuring the template. So for now we are having to use the console to re-configure after building the domain, and next we’ll use WLST to connect to a newly built domain and re-configure then bounce. I want an administration/configuration API that is not deprecated and is fully functional.

  Posted by: ejjohnson on August 6, 2007 at 8:33 AM

Those of you who read my blog would be aware that around late November of last year I decided our existing build process had to go. You can read my original rant here. Well I am please to report that after much discussion we bit the bullet a decided to scrap our build process and embark on creating a new process that we hoped would meet the increasing demands being made on the infrastructure team to build and deploy the numerous WLS domains that we manage. We have been using the new build process (which we call choom) for almost a year. In that time the demands to create new environments and support new versions of WebLogic, Apache and other middleware applications have only increased while our team has dropped in size from six to two people. If you are considering different approaches to building and deploying WLS domains I hope you find this and the next few entries useful. If you think these ideas are rubbish I want to know why?

To start with lets define some of the key requirments for our build process. The build process must:

• Create a complete WLS domain and scripts to distribute and install the domain onto multiple machines.
• Be fast. It should take no longer than a few minutes to build, distribute and install a WLS domain. This a key measure of the performance of a build process. The vast majority of WLS domains I have seen can be built and distrbuted in under a miniute on a desktop pc.
• Enforce architectural and design decisions and verify the configuration by checking the completed domain before it is distributed.
• Be simple to use. Anyone with basic unix skills should be able to build and deploy a domain after a 5 minute tutorial.
• Be able to build Apache, WebLogic and any other application maintained by the support team.
• Be simple to maintain. The build process should not require a team to support it. The process of adding new environments or applications must require no special skills.

With these requirements in place we invited BEA to help us evaluate different build approaches. The debates were always heated as everyone involved knew we had to get this right or we would end up in a mess. To outsiders it may well have seemed that the team was about to crack but in my experience civil and plesant technical discussion is often a sign that participants don’t know what they are talking about or don’t care. We whittled down the contenders to a) using WLST to dynamically create the domain and b) Create template domains and use ANT to “copy filter” the templates. We opted for ANT option for many reasons but the deciding reason was simple, WLST can only build WLS, the ANT option gave us a consistant method for building any application we wanted.

The next step was to create a template domain and define the properties. The customer had two sites, a live site and a contingency site. These sites were home to the production environment, a domain that spanned two machines. We would supply arguments to the build script which would determine which property files were loaded and used to create the domain. We had to group related properties together in their own files and we came up with the following files:

• common.properties – Used by all builds prefix=Common.
• mydomain.properties – Domain specific prefix=Domain
• production-live.properties – Production environment and live site prefix=Environment.
• production-cont.properties – Production environment and contingency site prefix=Environment.
• machine-live1.properties – Machine 1 in the live site prefix=Machine.
• machine-live2.properties – Machine 2 in the live site prefix=Machine.
• machine-cont1.properties – Machine 1 in the contingency site prefix=Machine.
• machine-cont2.properties – Machine 2 in the contingency site prefix=Machine.

To build the domain one would execute the following command:

ant -Dproduct=weblogic92 -Dsite=live -Ddomain=mydomain -Denv=production

The build loads the correct set of properties based on these arguments. This is a pretty standard approach familiar to anyone that uses ant, it looks like this:

<property name="properties.dir" location="${basedir}/properties"/>
<property file="${properties.dir}/common.properties"/>
<property file="${properties.dir}/domains/${domain}/${domain}.properties"/>
<property
file="${properties.dir}/domains/${domain}/environments/${env}/${product}/${env}-${site}.properties"/>

I’ll explain the directory structure in my next entry, to round off this entry I want to explain one of the problems with this standard ant approach and how we resolved it. If you have ever had to locate a property to change it’s value you have probably had a few hours of your life wasted by having to locate that property because it is defined in multiple files and ant can be configured to inherit properties. Our old build process didn’t have a consistant hierarchy which compounded the problem. It could take days to make a change to a single property! We were burned so badly by this that we decided there would be no inheritance of property files in Choom and that every property name would conform to a nomenclature that would indicate where that property was located in the build file system. For instance, if you were looking at a template of the config.xml you would see something like this:

    <ssl>
      <name>@Domain.admin.Server.Name@</name>
      <enabled>@Environment.admin.Server.SSL.Enabled@</enabled>

Any property prefixed with Domain corresponds to the domain level properties defined in mydomain.properties. If it’s prefixed with Environment you know it has to be in the production-${site}.property file for that site. Now we can locate and change the value of any property in seconds. For me this is a key metric to use in determining the performance of your build process. If you find it takes longer than a couple of minutes to make a simple configuration change in your build, alarm bells should start ringing.

In my next entry I will discuss the directory structure of choom and the templates. We gave alot of thought to the directiry structure of the build process. We wanted it to be intuative, consistant and to reflect how we perceivethe architecture our domains.

Before I get a flurry of comments about how the idea of having a single set of GC settings that is optimal for all Java applications is nonsense and farcical let me clarify what I mean by placing some limitations on this statement.

Firstly I will narrow the range of applications that I am talking about to J2EE applications built with a combination of JSPs, Servlets and EJBs, ok so I haven’t narrowed the range much.

Second I am not saying that these settings will be 100% optimal, but that they will probably be good enough. To quantify what I mean I would estimate that the number of major garbage collects over a say a 12 hour period while the application is under moderate to heavy load will be low and more importantly the percent of time spent by the JVM on this activilty will be a around 1% of the time it spends on all the other activity it is engaged in.

Finally these settings have been tested successfully in the field by a number of people and I have tested these setting myself against eight applications in the last year, all with very differernt memory footprints during volume and performance tuning phases. The results have always been very consistent, only today a couple of my colleagues independently came to settings that were very close to these “one size fits all” settngs.

Here they are:

-XX:NewRatio=4

Don’t specify the new size area using XX:NewSize as if you do change your heap size you will also have to remember to change this to keep the proportions allocated in your heap consistant. Specify a ratio instead, 4 is about right 2 it pointless as you will end up doing full GCs all the time.

-XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=80

You can play around with these but settings in order to find an optimum for your JVM. Make sure you use jvmstat to monitor the imapct of your experiments, the visualgc is useful for this purpose.

-XX:PermSize=128m -XX:MaxPermSize=128m

Unless your getting memory errors or your application is being profiled by a tools that rely on bytecode analysers you probably won’t need more than 128m of perm space. Use jvmstat to capture your applications total memory footprint.

-Xnoclassgc -XX:+DisableExplicitGC

You knows it makes sence to stop WLS or application code trying to incite unnecessary GC by calling GC directly. Leave GC it to the JVM!

I leave it up to the reader to find applications that make a mockery of these settings!
]]> http://www.genesis.uk.com/wordpress/archives/928/feed 8 http://www.genesis.uk.com/wordpress/archives/922 http://www.genesis.uk.com/wordpress/archives/922#comments Sun, 22 Nov 2009 14:06:20 +0000 husseinb http://www.genesis.uk.com/wordpress/?p=922 Hussein Badakhchani’s Blog April 12, 2006   5:27 AM

Have you ever wondered how a running WebLogic domain would cope with having the domain root directory deleted? Well if you have been responsible for resilience testing or excessively use rm -rf you may have some idea of the problems you could face if you loose your domain root.

I have witnessed some pretty disastrous effects on WLS 5 and WLS 6.1 domains as a result of deleting the domain root but a recent cleansing event on a weblogic 8.1 domain proved to me just how resilient this product has become.

I was asked by a developer to look at a print out of a log file which showed an EmbeddedLDAPException indicating that configuration files could not be found. This was an odd error but the obvious action was to check for the existence of the files, at this point I found that the entire domain was missing from the file system!

After checking the other machines in the cluster and finding the domain was missing from those machines as well I decided to try and identify exactly when the domain was deleted. Luckily our log files are located in a different location outside of the domain root, searching through the server logs it appeared that the domain had been deleted some three days earlier based on the first instance of the Embedded LDAP error I found. I was shocked by this, how was it possible that deleting a WLS domain from a working environment could go unnoticed for three days! Examining the log files further I found no other errors related to loss of the domain, it seemed that everything had continued to function correctly even after the deletion of the domain root directory.

Finally I decided to approach the developers working on ghost environment and explain the cause of the exception to them, they were as surprised as I was. We spent the next few hours double checking the application functionality as well as monitoring the domain using the console. I tested the JDBC connection pools, took thread dumps of the servers, examined the memory consumption and used WLST to dump the states of EJB MBeans, everything seemed to be o.k.

The developers continued their testing on the ghost integration domain for another seven days, the only limitation was that the servers could not be restarted, we could even deploy applications using the console.

Of course the application deserves some credit for not having any dependency on the domain file system but I was well impressed by WLS. We still don’t know who it was that deleted the domain in the first instance, an individual we have named “The ghost in the Korn shell” and to whom we have since attributed a number of other odd events, but at least now I know what happens if I delete the domain root directory. I wonder how JBoss and WebSphere would handle such events?

On a side note standard nerds and geeks may be interested to know that this entry was written using “Das Keyboard”.

Comments

Comments are listed in date ascending order (oldest first)

• If this has happened under *NIX environment (and it looks like it did), the reason WLS kept functioning may be the feature of the OS..

  OS keeps reference count on all open inodes (files or directories) and only TRULY DELETES stuff when all references are removed. So if WL server opened the files, you can go ahead and delete them – no-one will see them anymore, except for WL server, because it still holds reference to it. So WLS can read/write to these “ghost” files, but after the process exits, reference counts will go to zero and the files will be gone forever.

  Posted by: sLayer on April 12, 2006 at 10:09 AM

• I would certainly expect the OS not to kill the WLS processes just because the files they rely on were deleted, I’m not sure how WLS would behave under Windows although I do remember that Windows tends to lock files that are in use by a process.

  What impressed me though was the sheer lack of errors and the fact that appart from the LDAP exception every thing else seemed to be working fine. I can only explain this by assuming that much of the servers work is carried out in memory, and thats why removing these files had such little effect.

  Posted by: hoos on April 13, 2006 at 2:36 AM

• Great story Hoos! I agree with sLayer though – the files were probably not *really* deleted, but simply absent from your directory listings (and still available to the running processes). Still, that is neat.

  I think that keyboard is kinda funny. It sort of illustrates the difficult some folk have in training themselves not to look at the keyboard. So, now they can simply avoid the training by removing the keyboard – effectively doing the same thing at greater cost but with ease and absolutely no chance to cheat The strength zones sound interesting though. My pinky is powerless compared to my thunderous thumb!

  Posted by: jonmountjoy on April 13, 2006 at 5:10 AM

• I can now type without looking at the keyboard but its the shift key, punctuation and numbers that catch me out. Still I’ve only been using it for the last few days.

  Posted by: hoos on April 13, 2006 at 7:48 AM

• Actuall email recieved by me from Hoos on the first day of using Das Keyboard.
  
  >> HELP.I.CANT.FIND.THE.SPACE.BAR11

  
  
  My current client doesnt allow bring you own keyboard in, and my machine at home already has 2 keyboards attached (usb/wireless and ps/2 for Lilo) so my “das keyboard” is gathering dust.

  Posted by: simonvc on May 8, 2006 at 2:47 AM

• If only your current employers knew how you used to “build weblogic domains”, just make sure you don’t turn that green field into a mine field…

  On a more serious note, like some sort of crazy fool I bought the U.S version of “Das Keyboard” and as a result had to learn how to map the missing / and | key, in, erm, true hacker style I won’t let a missing key stop me!

  Posted by: hoos on May 8, 2006 at 8:28 AM

Building on my last entry on nomenclature I want to discuss the naming of environments. By an environment I mean all the infrastructure that belongs to a functional group of a project, for instance the User Acceptance Test environment or the Production environment.

A typical enterprise stack may consist of a Web Server, WebLogic server, perhaps a Digital Signature Messaging System or some Message Oriented Middleware like MQ and a database. Lets assume for simplicity that our environments have dedicated stacks, so my Functional Test environment is not sharing it’s database or any other infrastructure with my User Acceptance Test environment. What are the options for naming my environment? Well you could just give it an arbitrary name based on an abbreviation of it’s function and number to keep it unique. This is a fairly common practice and we end up with names like UAT01 and PRD01. Such a naming scheme is perfectly fine if you have a single project and you don’t reuse the environment for other projects but what if you have multiple projects and multiple environments, what problems can you expect with your basic naming scheme?

Well for a start there is no way of telling which environments belong to which projects from the name alone, “So what,” you might think and as you’re probably a miserable developer, who has lived inside an enclipse shell for the last couple of years producing very buggy code, your limited vision is unsurprising. Lets demonstrate the cost to the project whith this simple dialogue:

• Support Caller: Can we change the apache configuration for the Micky Mouse project.
• Support Engineer: Erm, ok what environment are you talking about?
• Support Caller: The test environment.
• Support Engineer: Which test environment, Functional Test or UAT?
• Support Caller: Functional Test.
• Support Engineer: We have 10 FT environments which one?
• Support Caller: Erm, I’ll get back to you on that.

Ok, this example maybe a little contrived and you’re probably thinking why doesn’t the defect tracker/ticketing system have all this information in it but it is a real scenario and a common one in meduim sized businesses. The point is we haven’t considered encoding the project name as metadata in the environment name and this piece of information is a key requirement not just for the support team but for anyone in the project that uses the environments.

As with naming your machines the first step in naming your environments is to identify your metadata requirements. In a recent exercise for one client we agreed on the following requirements:

1. The function of the environment should be present as a two character abbreviation e.g pd = Production, ft=Functional Test, ut = User Acceptance Test.
2. All environments that belong to the “Hafez” programme will be identified by the numeral 1.
3. The two projects in the Hafez programme, “Enterprise security” and “Processing engine” will be identified by the numerals 1 and 2 respectively.

We can express these requirements in Backus-Naur form:

<Environment Name> ::= <Environment Function><Programme ID><Project ID>

<Environment Function&gt ::= “FT” | “UT” | “PD”

<Programme ID> ::= “1″

<Project ID> ::= “1″ | “2″

These basic rules allowed us to name all the environments for the programme consistently, uniquely and in and easily identifiable fashion, for instance PD12 is the production environment for the processing engine project which is part of the Hafez programme.

While an environment naming convention is useful it really becomes poweful when we start to derive the names of key infrastructure artificates from our environment names. Lets list some of these artefacts and their derived names, we will stick with WebLogic artefacts for now. You may want to try wrtiting out the general rule for each artefact in Backus-naur form for yourself:

As you can see we have now consistently named listen addresses (hostnames), key stores and key aliases all from our environment name. Anyone can work out the name of the admin server for a given enironment based on our nomeclature. If you want to inspect the identity key you know what it’s alias should be. We can also derive names for database schemas, unix users and host of other infrastructure artefacts from the seed name, the environment name.

The benefits of a well thought out nomenclature are numerous and include:

1. Simplifies environment proving and automated validation of environments.
2. Simplifies the task of communicating and understanding the infrastructure architecture.
3. Maintains naming consistency across the programme.
4. Quicker creation of standardised specifications for new environments.
5. Facilitates automation of environment provisioning.

The last two points are where my interests lie, but if your going around touting yourself as an I.T Architect, I.T Technologist, I.T Architectural Technologist, guru, software imagineer, system-medic, Creative Management I.T Dude or some other exotic, bogus job title you should consider all the benefits before making up that random naming scheme on the plane back from some dodgy industry do or in a 10 minute teleconference with the offshore development team.

Comments

Comments are listed in date ascending order (oldest first) | Post Comment

• Hoos: I had a similar dialog with one of the developers for a weblogic domain and I had to log on to 2 different weblogic domains to find out which environment the developer is talking about. So having some sort of naming convention should improve the situation…

  Posted by: lalitkumar.dandekar on September 4, 2007 at 3:33 AM

• Lalit, another common scenario is this one:

  Support Caller 1: Can we change the Apache configuration for the Micky mouse project on uat.
  Support Engineer 1: Sure, done
  Support Caller 2: We are having problems on UAT03, we can’t reach the application.
  Support Engineer 2: Support Engineer 1 which UAT environment did you update?
  Support Engineer 1: Erm UAT…..

  You are right of course, for the nomeclature to be of any use all stake holders must be aware of it and of the environements they are using.

  Posted by: hoos on September 4, 2007 at 5:12 AM

#!/bin/bash

# ejbdump.sh
#
# author: Hussein Badakhchani
# date: 29 September 2005
# description: This scrip uses the weblogic.Admin tool to dump all data \
#              relating to a EJBMBeans
# processname: ejbdump
# info: Make sure your WebLogic environment is set up correctly. \
#       Run setEnv.sh before using ejbdump

prog=ejbdump

if [ ! $# == 4 ]; then
  echo
  echo "Usage: $0 url username password command_file"
  echo
  echo "e.g. ejbdump myhost:7001 myuser mypassword commands.txt"
  exit
fi

url=$1
username=$2
password=$3
command_file=$4

# Check for the existence of the command file.
if [ ! -f "$command_file" ]
then
  echo
  echo "Command file: $command_file not found!"
  echo "Please proivde the correct path to this file and ensure you have the \
correct privledges associated with it."
  exit $E_NOTFOUND
fi

java -cp $CLASSPATH weblogic.Admin -adminurl $url -username $username \
-password $password BATCHUPDATE -batchFile $command_file \
-continueOnError -batchCmdVerbose

get -pretty -type EJBCacheRuntime
get -pretty -type EJBComponentRuntime
get -pretty -type EJBLockingRuntime
get -pretty -type EJBPoolRuntime
get -pretty -type EJBRuntime
get -pretty -type EJBTransactionRuntime
get -pretty -type EntityCacheCumulativeRuntime
get -pretty -type EntityCacheCurrentStateRuntime
get -pretty -type EntityCacheRuntime
get -pretty -type EntityEJBRuntime

[husseinb@lap1 bea]$ ./ejbdump lap3:7001 weblogic weblogic commands.txt

Executing command: get -pretty -type EntityCacheRuntime

No MBeans found

Error executing command in batch file commands.txt at line number: 9

Executing command: get -pretty -type EntityEJBRuntime

---------------------------
MBeanName: "examples:ApplicationRuntime=examplesServer__appsdir_ejb20_basic_containerManaged_ear,
EJBComponentRuntime=examplesServer__appsdir_ejb20_basic_containerManaged_ear_ejb20_basic_containerManaged.jar,
Location=examplesServer,
Name=examplesServer__appsdir_ejb20_basic_containerManaged_ear_ejb20_basic_containerManaged.jar_containerManaged,
ServerRuntime=examplesServer,Type=EntityEJBRuntime"
        CacheRuntime: examplesServer__appsdir_ejb20_basic_containerManaged_ear_ejb20_basic_containerManaged.jar_containerManaged
        CachingDisabled: true
        EJBName: containerManaged
        LockingRuntime:
        Name: examplesServer__appsdir_ejb20_basic_containerManaged_ear_ejb20_basic_containerManaged.jar_containerManaged
        ObjectName: examplesServer__appsdir_ejb20_basic_containerManaged_ear_ejb20_basic_containerManaged.jar_containerManaged

        Parent: examplesServer__appsdir_ejb20_basic_containerManaged_ear_ejb20_basic_containerManaged.jar
        PoolRuntime: examplesServer__appsdir_ejb20_basic_containerManaged_ear_ejb20_basic_containerManaged.jar_containerManaged
        Registered: false
        Resources:
        TransactionRuntime: examplesServer__appsdir_ejb20_basic_containerManaged_ear_ejb20_basic_containerManaged.jar_containerManaged
        Type: EntityEJBRuntime
---------------------------
MBeanName:"examples:ApplicationRuntime=examplesServer__appsdir_ejb20_basic_beanManaged_ear,
EJBComponentRuntime=examplesServer__appsdir_ejb20_basic_beanManaged_ear_ejb20_basic_beanManaged.jar,
Location=examplesServer,
Name=examplesServer__appsdir_ejb20_basic_beanManaged_ear_ejb20_basic_beanManaged.jar_beanManaged,
ServerRuntime=examplesServer,Type=EntityEJBRuntime"
        CacheRuntime: examplesServer__appsdir_ejb20_basic_beanManaged_ear_ejb20_basic_beanManaged.jar_beanManaged
        CachingDisabled: true
        EJBName: beanManaged
        LockingRuntime:
        Name: examplesServer__appsdir_ejb20_basic_beanManaged_ear_ejb20_basic_beanManaged.jar_beanManaged
        ObjectName: examplesServer__appsdir_ejb20_basic_beanManaged_ear_ejb20_basic_beanManaged.jar_beanManaged
        Parent: examplesServer__appsdir_ejb20_basic_beanManaged_ear_ejb20_basic_beanManaged.jar
        PoolRuntime: examplesServer__appsdir_ejb20_basic_beanManaged_ear_ejb20_basic_beanManaged.jar_beanManaged
        Registered: false
        Resources:
        TransactionRuntime: examplesServer__appsdir_ejb20_basic_beanManaged_ear_ejb20_basic_beanManaged.jar_beanManaged
        Type: EntityEJBRuntime

---------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------
Batch Command Results:
Total Commands Executed: 10
Commands Successful: 6
Commands Failed: 4

java -cp $CLASSPATH weblogic.Admin -url lap3:7001 -username weblogic \
-password weblogic GET -pretty -mbean \
examples:ApplicationRuntime=examplesServer__appsdir_ejb20_basic_containerManaged_ear,
EJBComponentRuntime=examplesServer__appsdir_ejb20_basic_containerManaged_ear_ejb20_basic_containerManaged.jar,
Location=examplesServer,
Name=examplesServer__appsdir_ejb20_basic_containerManaged_ear_ejb20_basic_containerManaged.jar_containerManaged,
ServerRuntime=examplesServer,Type=EJBTransactionRuntime