TIBCO is a company that provides enterprise solutions for Analytics, Cloud, Customer Loyalty, Event Processing and Integration.
I am specialised in using their Integration tool called Business Works (BW).
Tibco BW has irreverently been called the gaffer tape that glues applications together.
In its basic form BW is used for file transfer and simple Rendezvous messaging. In its most robust incarnation it is used as a full Service Orientated Architecture (SOA).
In 2008 the insurance company that I was working at were preparing to migrate their proprietary SOA to a TIBCO SOA. I took some courses (‘BusinessWorks 5.x Integration Boot Camp’ and ‘Essentials of TIBCO BPM 10.x’) and got involved by doing a proof of concept to migrate the proprietary mainframe interfaces to TIBCO, combining my mainframe experience with newly acquired TIBCO skills.
I was responsible for the technical design of the CICS interfaces (via SubStation) and the batch interfaces (via native EMS). I built the proof of concept interfaces. (See ‘Technical details about the Mainframe integration project’ below.)
I built the templates for the various interface patterns after which the other team members would build the interfaces using the template.
From the start of the project until October 2013 I functioned as lead developer for the mainframe and TIBCO developers. At one time the team had seven developers. From July till October 2013 I also had the role of Scrum Master.
After completing my part of the project I left at the end of October 2013.
In November 2013 I started as a Senior TIBCO developer on a project at a merchant bank in Amsterdam Zuid.
The purpose of the project was to implement a new front-end trading system (thinkFolio) to replace the old Latent Zero system.
I did the Tibco development work for a project to replace the front-end system for the investment bank. The bare bones of the Tibco application was in place. I completed the build and then re-engineered and refactored the application to make it stable and make it run faster and more efficiently.
The Tibco application feeds the ThinkFolio front-end system with a daily load. Before the refactoring the daily load was very unstable, would run for four hours and the whole load would have to be restarted if it failed. After the refactoring (and some new hardware) the load runs in one and a half hours and is very stable. The load can be restarted from the last point of failure, shortening the time till when the bank is open for trading.
I am currently working as a software engineer on a project to implement a Markit EDM data hub. I did a lot of analysis of the existing Tibco applications to reverse engineer some of the requirements for the data hub. One of the requirements of the project is that the existing Tibco applications must remain untouched during the migration to the new data hub. The old data ‘hub’ is called by the Tibco applications using Rendezvous services. I designed and built a Tibco application with which the existing services are converted to SOAP services that can query the EDM data hub. The Tibco applications remain untouched while the data hub is migrated.
********************
Technical details about the Mainframe integration project at the insurance company
TIBCO tooling:
TIBCO SubStation
TIBCO EMS API
TIBCO EMS queues
TIBCO BusinessWorks
TIBCO AMX
TIBCO AMX Admin console
Knowledge of xml, xsd and wsdl were required.
The work required not only in depth knowledge of the above mentioned TIBCO tooling, but also of the z/OS mainframe ecosystem.
The project aimed to migrate 190 interfaces from a home-grown SOA (called IFSA) to TIBCO.
The interfaces communicate via MQ queues. Interfaces are of the types Fire&Forget or Request/Reply. Message are produced and consumed by the mainframe applications. The mainframe uses the interfaces in both CICS (on-line) and Batch.
For IFSA, the transformation from COBOL attributes to xml is done with XML Composer on the mainframe. In TIBCO, the transformation is done in TIBCO BusinessWorks.
TIBCO supplies documentation on how to create the adapters for TIBCO as well as working samples. Unfortunately there is a big gap between their working samples and an adapter that does something useful for your mainframe applications.
This meant that the adapters had to be designed and customised to work with the existing applications.
I designed the various adapter types that we would then use generically for the migration.
CICS via SubStation adapters:
Request/Reply message consumer
Request/Reply message producer
Fire & Forget message consumer
Fire & Forget message producer
Batch via EMS API:
Request/Reply message producer
Fire & Forget message producer
Fire & forget message consumer (Triggered by CICS)
The batch adapters were particularly difficult to get working. We had on site support form TIBCO, but it still took weeks to get the first one up and running.
Once the adapters on the mainframe side were working, the TIBCO BusinessWorks (BW) components needed to be designed and built. The first iteration was based on TIBCO best practices using the TIBCO COBOLCopyBook Plugin.
We then refined the the design to handle the mainframe adapter types. A colleague wrote a generator in Python with which new BW adapters could be generated. this standardised the adapters and is great for the guys who have to maintain the adapters in future. The generated adapter contains the standard logging components we created. To complete an adapter, the specific copybook-to-xml mappings need to be added.
In BW there are four patterns:
Request/Reply message consumer
Request/Reply message producer
Fire & Forget message consumer
Fire & Forget message producer
After unit testing the BW adapters, the BW project is brought into an AMX project. The BW adapters are built into composites. After mapping the BW adapters to the wsdls and configuring a hundred other things, a DAA file can be built.
The DAA file is deployed onto a server using the TIBCO AMX Admin console.
To make the complete chain work smoothly, various things need to be configured:
TIBCO SubStation Recipes and Triggers
EMS P2P queues between SubStation and BW and between the native COBOL and BW
Grants on the P2P queues
JCL’s for batch programs
EMS ESB queues to and from AMX
Grants on ESB queues
Now that all the bits had been assembled they needed to be tested. With my knowledge of both TIBCO and mainframe, I could monitor the whole chain from the originating application, through EMS, AMX, BW right onto the mainframe.
I did the implementations of the mainframe components to production.