Friday, 22 September 2017

IBM Installation Manager and the Not-Well-Formed Markup

I saw this: -

ERROR: Failed to read response file.
  ERROR: Problem in /mnt/installIIM186.rsp at line 5: The markup in the document following the root element must be well-formed.
00:00.52 ERROR [main] com.ibm.cic.agent.core.application.HeadlessApplication run
  Failed to read response file.
    Problem in /mnt/installIIM186.rsp at line 5: The markup in the document following the root element must be well-formed.


whilst trying to install IBM Installation Manager 1.8.7, using a response file: -

/mnt/installIIM187.rsp

<?xml version="1.0" encoding="UTF-8"?>
<server>
<repository location='/mnt/IM64' temporary="true"/>
</server>
<profile id='IBM Installation Manager' installLocation='/opt/ibm/InstallationManager/eclipse' kind='self'>
<data key='eclipseLocation' value='/opt/ibm/InstallationManager/eclipse'/>
<data key='user.import.profile' value='false'/>
<data key='cic.selector.os' value='linux'/>
<data key='cic.selector.arch' value='x86_64'/>
<data key='cic.selector.ws' value='gtk'/>
<data key='cic.selector.nl' value='de,no,fi,ru,hr,fr,hu,sk,sl,sv,ko,el,en,pt_BR,it,iw,zh,es,cs,ar,zh_HK,zh_TW,th,ja,pl,da,tr,nl'/>
</profile>
<install modify='false'>
<offering id='com.ibm.cic.agent' version='1.8.7000.20170706_2137' profile='IBM Installation Manager' features='agent_core,agent_jre' installFixes='none'/>
</install>
<preference name='com.ibm.cic.common.core.preferences.eclipseCache' value='/opt/ibm/IMShared'/>
<preference name='com.ibm.cic.common.core.preferences.connectTimeout' value='30'/>
<preference name='com.ibm.cic.common.core.preferences.readTimeout' value='45'/>
<preference name='com.ibm.cic.common.core.preferences.downloadAutoRetryCount' value='0'/>
<preference name='offering.service.repositories.areUsed' value='true'/>
<preference name='com.ibm.cic.common.core.preferences.ssl.nonsecureMode' value='false'/>
<preference name='com.ibm.cic.common.core.preferences.http.disablePreemptiveAuthentication' value='false'/>
<preference name='http.ntlm.auth.kind' value='NTLM'/>
<preference name='http.ntlm.auth.enableIntegrated.win32' value='true'/>
<preference name='com.ibm.cic.common.core.preferences.preserveDownloadedArtifacts' value='true'/>
<preference name='com.ibm.cic.common.core.preferences.keepFetchedFiles' value='false'/>
<preference name='PassportAdvantageIsEnabled' value='false'/>
<preference name='com.ibm.cic.common.core.preferences.searchForUpdates' value='false'/>
<preference name='com.ibm.cic.agent.ui.displayInternalVersion' value='false'/>
<preference name='com.ibm.cic.common.sharedUI.showErrorLog' value='true'/>
<preference name='com.ibm.cic.common.sharedUI.showWarningLog' value='true'/>
<preference name='com.ibm.cic.common.sharedUI.showNoteLog' value='true'/>
</agent-input>


I dug around, and found out where I'd gone wrong.

I needed to have the line: -

<agent-input acceptLicense='true'>

as per this: -

<?xml version="1.0" encoding="UTF-8"?>
<agent-input acceptLicense='true'>
<server>
<repository location='/mnt/IM64' temporary="true"/>
</server>
<profile id='IBM Installation Manager' installLocation='/opt/ibm/InstallationManager/eclipse' kind='self'>
<data key='eclipseLocation' value='/opt/ibm/InstallationManager/eclipse'/>
<data key='user.import.profile' value='false'/>
<data key='cic.selector.os' value='linux'/>
<data key='cic.selector.arch' value='x86_64'/>
<data key='cic.selector.ws' value='gtk'/>
<data key='cic.selector.nl' value='de,no,fi,ru,hr,fr,hu,sk,sl,sv,ko,el,en,pt_BR,it,iw,zh,es,cs,ar,zh_HK,zh_TW,th,ja,pl,da,tr,nl'/>
</profile>
<install modify='false'>
<offering id='com.ibm.cic.agent' version='1.8.7000.20170706_2137' profile='IBM Installation Manager' features='agent_core,agent_jre' installFixes='none'/>
</install>
<preference name='com.ibm.cic.common.core.preferences.eclipseCache' value='/opt/ibm/IMShared'/>
<preference name='com.ibm.cic.common.core.preferences.connectTimeout' value='30'/>
<preference name='com.ibm.cic.common.core.preferences.readTimeout' value='45'/>
<preference name='com.ibm.cic.common.core.preferences.downloadAutoRetryCount' value='0'/>
<preference name='offering.service.repositories.areUsed' value='true'/>
<preference name='com.ibm.cic.common.core.preferences.ssl.nonsecureMode' value='false'/>
<preference name='com.ibm.cic.common.core.preferences.http.disablePreemptiveAuthentication' value='false'/>
<preference name='http.ntlm.auth.kind' value='NTLM'/>
<preference name='http.ntlm.auth.enableIntegrated.win32' value='true'/>
<preference name='com.ibm.cic.common.core.preferences.preserveDownloadedArtifacts' value='true'/>
<preference name='com.ibm.cic.common.core.preferences.keepFetchedFiles' value='false'/>
<preference name='PassportAdvantageIsEnabled' value='false'/>
<preference name='com.ibm.cic.common.core.preferences.searchForUpdates' value='false'/>
<preference name='com.ibm.cic.agent.ui.displayInternalVersion' value='false'/>
<preference name='com.ibm.cic.common.sharedUI.showErrorLog' value='true'/>
<preference name='com.ibm.cic.common.sharedUI.showWarningLog' value='true'/>
<preference name='com.ibm.cic.common.sharedUI.showNoteLog' value='true'/>
</agent-input>


/mnt/IM64/tools/imcl -input /mnt/installIIM187.rsp -acceptLicense

Installed com.ibm.cic.agent_1.8.7000.20170706_2137 to the /opt/ibm/InstallationManager/eclipse directory.

Job done :-)

IBM Business Process Manager 8.6

As per my previous post: -


IBM BPM 8.6 was released today, and I've started the download.

This is what I'm pulling down as I type: -

IBM Business Process Manager Server Version 8.6 For Linux X86 64Bit Multilingual (3 of 3) (CNM6BML )

IBM Business Process Manager Server Version 8.6 For Linux X86 64Bit Multilingual (2 of 3) (CNM6AML )

IBM Business Process Manager Server Version 8.6 For Linux X86 64Bit Multilingual (1 of 3) (CNM69ML )


More to come …

Wednesday, 20 September 2017

Kubernetes 1.7 available in IBM Bluemix Container Service

This arrived in my inbox today: -

We're excited to announce that Kubernetes 1.7 is available for IBM Bluemix Container Service. You can now update your Kubernetes master and worker nodes to the latest supported version of Kubernetes by using either the Bluemix dashboard or the CLI.


This is perfect timing for me, as: -

(a) I'm reading and reviewing Kubernetes Microservices with Docker 
(b) I've been tinkering with DB2 and WebSphere Liberty Profile on Docker and Kubernetes, in preparation for an upcoming Lunch and Learn "brown bag" session that I'm delivering to my Services team.

Can you say "Awesome" ?

Using openSSL on macOS to encrypt a file using a password

I had a requirement to share a file with a colleague, which I did using Box. However, I wanted to go one step further and encrypt the file BEFORE sharing.

This is known, in some circles, as Pre-Internet Encryption (PIE), which is funny, because I like pie - fish pie, apple pie, mince pie, you name it :-)

This is what I did: -

Encrypt the file

openssl enc -aes-256-cbc -in Patent.doc > Patent_enc.doc 

This example uses the AES-256-CBC cipher and requests a password, which is used, with the chosen block/stream cipher, to encrypt the file.

My colleague then used a similar command: -

openssl enc -aes-256-cbc -in Patent_enc.doc -d > Patent.doc 

to decrypt the file.

I could've used one of a number of ciphers: -



For the record, whilst I shared the file with him via Box, I shared the decryption command via Slack, and the password via a third, separate channel.

Don't call me paranoid :-)

From the Wiki here: -

This page describes the command line tools for encryption and decryption. Enc is used for various block and stream ciphers using keys based on passwords or explicitly provided. It can also be used for Base64 encoding or decoding.

It's also worth noting that the openSSL command on macOS is somewhat limited / out-of-date.

This is what I have: -

openssl version

OpenSSL 0.9.8zh 14 Jan 2016

as compared to Red Hat: -

openssl version

OpenSSL 1.0.1e-fips 11 Feb 2013

So the macOS version is older but newer :-)

As an example, this command ( from the Wiki ): -

openssl list-cipher-algorithms

gives this error on macOS: -

openssl:Error: 'list-cipher-algorithms' is an invalid command.

whereas on RHEL, it gives a huge list :-)

However, I was able to work out what ciphers the command supported: -

openssl help

openssl:Error: 'help' is an invalid command.

Standard commands
asn1parse      ca             ciphers        crl            crl2pkcs7      
dgst           dh             dhparam        dsa            dsaparam       
ec             ecparam        enc            engine         errstr         
gendh          gendsa         genrsa         nseq           ocsp           
passwd         pkcs12         pkcs7          pkcs8          prime          
rand           req            rsa            rsautl         s_client       
s_server       s_time         sess_id        smime          speed          
spkac          verify         version        x509           

Message Digest commands (see the `dgst' command for more details)
md2            md4            md5            mdc2           rmd160         
sha            sha1           

Cipher commands (see the `enc' command for more details)
aes-128-cbc    aes-128-ecb    aes-192-cbc    aes-192-ecb    aes-256-cbc    
aes-256-ecb    base64         bf             bf-cbc         bf-cfb         
bf-ecb         bf-ofb         cast           cast-cbc       cast5-cbc      
cast5-cfb      cast5-ecb      cast5-ofb      des            des-cbc        
des-cfb        des-ecb        des-ede        des-ede-cbc    des-ede-cfb    
des-ede-ofb    des-ede3       des-ede3-cbc   des-ede3-cfb   des-ede3-ofb   
des-ofb        des3           desx           rc2            rc2-40-cbc     
rc2-64-cbc     rc2-cbc        rc2-cfb        rc2-ecb        rc2-ofb        
rc4            rc4-40         seed           seed-cbc       seed-cfb       
seed-ecb       seed-ofb       


Tuesday, 19 September 2017

This time, it's about a freezer

So almost all of my blog posts are technical, and most involve some kind of IT and/or IBM product or service.,

This time, whilst still technology, it's all about …. freezers.

We recently took delivery of a Zanussi ZFT10210WA freezer, and hit a problem ….

Specifically, it was a UI problem.

More specifically, the UI didn't match the documentation ( available as a PDF here ).

This is what the documentation has: -


whereas the freezer looks more like this: -


In other words, how can I set it to -16 degrees C when the Temperature Regulator knob only shows 1-6 ?

I tried Zanussi's support page: -


but they don't actually list freezers there: -

so I tried the email address on the page: -


which bounced back.

I also tried the other email address on the page ( hover over one and the other one is revealed below ): -


but that also bounced back.

Thankfully, I found a Twitter page for @Zanussi_UK  which, despite not having much activity since June 2016, did include a Tweet with an old email address: -


I emailed this address: -


and they came straight back with this: -

<snip>
Thank you for your email below, I'm sorry that the user manual is not showing the correct information.
 
I can confirm that the temperature control within the freezer section should be set  between 3-4 on the dial, this will reduce the temperature to between -16 and -18 degrees.

 </snip>

which is nice.

Thankfully, I already had the dial set midway between 3 and 4, which was a lucky guess.

So, the moral of the story ?

Try the web, try email, try Twitter, and then try email again :-)

Kubernetes and IBM Bluemix - again with the #HoldingItWrong

So I saw this: -

kubectl get nodes

Unable to connect to the server: could not refresh token: unrecognized error {"errorCode":"BXNIM0408E","errorMessage":"Provided refresh token is expired","context":{"requestId":"4294322993","requestType":"incoming.Kube_Token","startTime":"19.09.2017 11:58:26:739 UTC","endTime":"19.09.2017 11:58:26:741 UTC","elapsedTime":"2","instanceId":"tokenservice_dal06/1","host":"localhost","threadId":"1955e0","clientIp":"146.90.214.170","userAgent":"Go-http-client/1.1","locale":"en_US"}}

which made me realise that I had forgotten to set the KUBECONFIG environment variable ( I'm using macOS ).

This I did: -

export KUBECONFIG=/Users/davidhay/.bluemix/plugins/container-service/clusters/DaveHayK8SCluster/kube-config-prod-dal10-DaveHayK8SCluster.yml
echo $KUBECONFIG


/Users/davidhay/.bluemix/plugins/container-service/clusters/DaveHayK8SCluster/kube-config-prod-dal10-DaveHayK8SCluster.yml

but I then started getting this: -

kubectl get nodes

The connection to the server localhost:8080 was refused - did you specify the right host or port?

kubectl proxy

The connection to the server localhost:8080 was refused - did you specify the right host or port?

which made me cuss a bit.

However …..

I love it when my own blog post: -


has the solution :-)

Bottom line, the KUBECONFIG variable was AGAIN wrong :-(

I validated this: -

ls $KUBECONFIG

ls: /Users/davidhay/.bluemix/plugins/container-service/clusters/DaveHayK8SCluster/kube-config-prod-dal10-DaveHayK8SCluster.yml: No such file or directory

So I validated the name of my Kubernetes cluster: -

bx cs clusters

OK
Name                ID                                 State    Created                    Workers   Datacenter   
DaveHayK8SCluster   fff102198c534d0096bacd575488c9dd   normal   2017-08-21T09:59:53+0000   1         par01   


and then searched for the YAML: -

find ~/.bluemix/ -name *.yml

/Users/davidhay/.bluemix//plugins/container-service/clusters/DaveHay/kube-config-prod-dal10-DaveHay.yml
/Users/davidhay/.bluemix//plugins/container-service/clusters/DaveHayK8SCluster/kube-config-par01-DaveHayK8SCluster.yml


Once i set the variable appropriately: -

export KUBECONFIG=/Users/davidhay/.bluemix//plugins/container-service/clusters/DaveHayK8SCluster/kube-config-par01-DaveHayK8SCluster.yml

things started working again: -

kubectl get nodes

NAME            STATUS    AGE       VERSION
10.127.239.36   Ready     29d       v1.5.6-4+abe34653415733

kubectl proxy

Starting to serve on 127.0.0.1:8001

and the proxy now works: -


So again, READ MY (OWN) BLOG :-)

Monday, 18 September 2017

Testing JDBC Data Sources using Jython

One of my colleagues asked me about this …

In essence, did I have a Jython script that allows one to test JDBC data source …

Here's one I prepared earlier: -

testDataSource.jy

cellID = AdminControl.getCell()
cell=AdminConfig.getid( '/Cell:'+cellID+'/')
for dataSource in AdminConfig.list('DataSource',cell).splitlines():
 print dataSource
 AdminControl.testConnection(dataSource)


Notes: -

- To support the FOR loop, there are indentations ( thanks Python, we love you ) in front of the last two lines of the script
- Similarly, there's a spare, blank line ( again, thanks, Python ) at the end of the script to finish the loop

When I run this: -

I do get an exception, for which I'm NOT catching: -


specifically this: -

DefaultEJBTimerDataSource(cells/PCCell1/applications/commsvc.ear/deployments/commsvc|resources.xml#DataSource_1228749623069)
WASX7017E: Exception received while running file "testDataSource.jy"; exception information: com.ibm.websphere.management.exception.AdminException
javax.management.MBeanException
java.sql.SQLException: java.sql.SQLException: Database '/opt/ibm/WebSphereProfiles/AppSrv01/databases/EJBTimers/AppClusterMember1/EJBTimerDB' not found. DSRA0010E: SQL State = XJ004, Error Code = 40,000


I could mitigate that by adding the appropriate try/catch logic to my script - that's tomorrow's challenge.

For the record, this exception occurs against a datasource about which I don't care :-)

Secure Identity Propagation Using WS-Trust, SAML2, and WS-Security

I'm reading this: -


in the context of Single Sign-on (SSO), via this: -


and: -


Friday, 15 September 2017

New Technology Demonstration: BPM Analytics

This from my IBM colleague, Allan Chan: -

A new BPM Analytics technology demonstration is available to use with the latest IBM Business Process Manager. The latest version works with V8.5.7.0 CF201706 release at the end of June 2017. The original version worked with V8.5.7.0 CF201703 released on 31st March 2017.
...
The key value of IBM Business Process Manager (BPM) is in streamlining custom enterprise business processes to better optimize service and cost. It does this namely through 1) custom process applications to manage work, and 2) process analytics for workers, managers, and analysts to assist their decision making in the execution and management of work and the design of processes. This technology demonstration aims to enhance BPM capabilities for (2) process analytics, with modern technologies which can excel in the era of big data and analytics, taking advantage of the full potential of the rich information into business operations afforded by custom process apps executing in BPM.

The BPM Analytics aims to enhance BPM for two scenarios: 1) BPM Analytics – providing enhanced process analytics features directly within the BPM offering targeting BPM user roles, and 2) 3rd Party analytics – providing enhanced features to publish process data to external data and analytics solutions provided by IBM, customers, and partners.

Monday, 11 September 2017

IBM Redbook - Developing Node.js Applications on IBM Bluemix


This IBM® Redbooks® publication explains how to create various applications based on Node.js and run them on IBM Bluemix®. In this book, you will do the following activities: 

• Develop a Hello World application in Node.js, executing on IBM Bluemix. Through this activity, you can learn about these technologies:

• IBM SDK for Node.js 
• Eclipse Orion Web IDE 

• Use asynchronous callback
• Create an Express application
• Build a rich user interface application by using AngularJS based in Node.js

This book is for beginner and experienced developers who want to start coding Node.js applications on IBM Bluemix.

Table of contents

Chapter 1. Developing a Hello World Node.js app on Bluemix
Chapter 2. Understanding asynchronous callback
Chapter 3. Creating your first express application
Chapter 4. Building a rich UI application by using AngularJS with Node.js
Appendix A. Additional material

IBM API Connect - new newness

From here: -



which is MERELY a subset of what's changed.

So go and have a look …

And/or download the update from IBM Fix Central here.

Thursday, 7 September 2017

Node-RED on IBM Bluemix - Deleting Wires

So this caused me grief for a brief period, until I turned to Google :-)

I am editing a flow in Node-RED on IBM Bluemix 


and wanted to delete a connection ( line ) between two nodes, as illustrated above.

Now how the heck can I do this ? I tried clicking the right-hand mouse button


( I'm using Chrome )

but that wasn't too useful.

I tried double-clicking on the offending connection .. no dice

Then I turned to Google and found this: -


Hold down shift while dragging from the end of the wire you want to move

In essence, I held down the shift key and dragged the connection away from the node and … let go.


Simples :-)

Wednesday, 30 August 2017

Tinkering with IBM Containers on IBM Bluemix - Like Docker really ...

As per previous posts, I have been tinkering with IBM Containers (IC) on IBM Bluemix, and am starting to bring things together in my mind, in terms of positioning IC vs. Kubernetes vs. Docker.

One of the many things that I like is the amount of help and choices that one has.

Having initialised the Bluemix ( bc ) Cloud Foundry ( cf ) IBM Containers ( ic ) environment: -

bx cf ic init

Invoking 'cf ic init'...

Deleting old configuration file...
OK
Generating client certificates for IBM Containers...
Client certificates are being stored in /Users/foobar/.ice/certs/...

Client certificates are being stored in /Users/foobar/.ice/certs/containers-api.eu-gb.bluemix.net/21377cbf-6e5f-4a9a-175a-4fdfeb3c3e12...

OK
Client certificates were retrieved.

Checking local Docker configuration...
OK

Authenticating with registry at host name registry.eu-gb.bluemix.net
OK
You are authenticated with IBM Bluemix Container Registry.
Your private Bluemix repository is URL: registry.eu-gb.bluemix.net/foobar
No ic-cfg.ini found on the system. Creating...

You can choose from two ways to use the Docker CLI with IBM Containers:


Option 1: This option allows you to use 'cf ic' for managing containers on IBM Containers while still using the Docker CLI directly to manage your local Docker host.
Use this Cloud Foundry IBM Containers plug-in without affecting the local Docker environment:


Example Usage:
cf ic ps
cf ic images

Option 2: Use the Docker CLI directly. In this shell, override the local Docker environment to connect to IBM Containers by setting these variables. Copy and paste the following commands:
Note: Only Docker commands followed by (Docker) are supported with this option. 
  export DOCKER_HOST=tcp://containers-api.eu-gb.bluemix.net:8443
  export DOCKER_CERT_PATH=/Users/foobar/.ice/certs/containers-api.eu-gb.bluemix.net/21377cbf-6e5f-4a9a-175a-4fdfeb3c3e12
  export DOCKER_TLS_VERIFY=1

Example Usage:
docker ps
docker images



it's great to see that I can use either cf ic commands: -

cf ic ps

CONTAINER ID        IMAGE                                                       COMMAND             CREATED             STATUS              PORTS                            NAMES
3cf533af-c95        registry.eu-gb.bluemix.net/foobar/db2expressc:pamfixed   "db2start "         8 days ago          Running             134.168.59.83:50000->50000/tcp   db2


cf ic images

REPOSITORY                                               TAG                 IMAGE ID            CREATED             SIZE
registry.eu-gb.bluemix.net/ibmnode                       v1.2                b1667ce7e5af        2 weeks ago         183MB
registry.eu-gb.bluemix.net/ibmnode                       v1.1                18f8f073b62b        2 weeks ago         176MB
registry.eu-gb.bluemix.net/ibm-websphere-extreme-scale   latest              8fccb460321a        7 weeks ago         466MB
registry.eu-gb.bluemix.net/ibm-integration-bus           latest              4b5f5fb39008        4 weeks ago         698MB
registry.eu-gb.bluemix.net/ibm_wa_agent                  latest              db7dc2abff64        4 months ago        435MB
registry.eu-gb.bluemix.net/ibm-mq                        latest              b42d55b53b18        8 days ago          455MB
registry.eu-gb.bluemix.net/ibmliberty                    webProfile7         cd826253309d        7 weeks ago         268MB
registry.eu-gb.bluemix.net/ibmnode                       v4                  20020df7341b        2 weeks ago         188MB
registry.eu-gb.bluemix.net/ibmnode                       latest              20020df7341b        2 weeks ago         188MB
registry.eu-gb.bluemix.net/ibm-backup-restore            latest              ea90ec45b636        4 weeks ago         205MB
registry.eu-gb.bluemix.net/ibm-node-strong-pm            latest              9499f23eb689        4 months ago        281MB
registry.eu-gb.bluemix.net/ibmliberty                    webProfile6         b350d9fd08fd        7 weeks ago         260MB
registry.eu-gb.bluemix.net/ibmliberty                    latest              edea51e84600        7 weeks ago         307MB
registry.eu-gb.bluemix.net/ibmliberty                    javaee7             edea51e84600        7 weeks ago         307MB
registry.eu-gb.bluemix.net/ibmliberty                    microProfile        4a50d0f49c6e        7 weeks ago         230MB
registry.eu-gb.bluemix.net/foobar/db2expressc         pamfixed            968890871f71        8 days ago          640MB

 -OR- docker commands: -

docker ps

CONTAINER ID        IMAGE                                                       COMMAND             CREATED             STATUS              PORTS                            NAMES
3cf533af-c95        registry.eu-gb.bluemix.net/foobar/db2expressc:pamfixed   "db2start "         8 days ago          Running             134.168.59.83:50000->50000/tcp   db2


docker images

REPOSITORY                                               TAG                 IMAGE ID            CREATED             SIZE
registry.eu-gb.bluemix.net/ibm-integration-bus           latest              4b5f5fb39008        4 weeks ago         698MB
registry.eu-gb.bluemix.net/ibm_wa_agent                  latest              db7dc2abff64        4 months ago        435MB
registry.eu-gb.bluemix.net/ibm-websphere-extreme-scale   latest              8fccb460321a        7 weeks ago         466MB
registry.eu-gb.bluemix.net/ibm-mq                        latest              b42d55b53b18        8 days ago          455MB
registry.eu-gb.bluemix.net/ibm-node-strong-pm            latest              9499f23eb689        4 months ago        281MB
registry.eu-gb.bluemix.net/ibmnode                       latest              20020df7341b        2 weeks ago         188MB
registry.eu-gb.bluemix.net/ibmnode                       v4                  20020df7341b        2 weeks ago         188MB
registry.eu-gb.bluemix.net/ibmnode                       v1.1                18f8f073b62b        2 weeks ago         176MB
registry.eu-gb.bluemix.net/ibmliberty                    webProfile7         cd826253309d        7 weeks ago         268MB
registry.eu-gb.bluemix.net/ibmliberty                    microProfile        4a50d0f49c6e        7 weeks ago         230MB
registry.eu-gb.bluemix.net/ibmliberty                    latest              edea51e84600        7 weeks ago         307MB
registry.eu-gb.bluemix.net/ibmliberty                    javaee7             edea51e84600        7 weeks ago         307MB
registry.eu-gb.bluemix.net/ibmliberty                    webProfile6         b350d9fd08fd        7 weeks ago         260MB
registry.eu-gb.bluemix.net/ibm-backup-restore            latest              ea90ec45b636        4 weeks ago         205MB
registry.eu-gb.bluemix.net/ibmnode                       v1.2                b1667ce7e5af        2 weeks ago         183MB
registry.eu-gb.bluemix.net/foobar/db2expressc         pamfixed            968890871f71        8 days ago          640MB

You have a choice - and choices are good :-)

Friday, 25 August 2017

IBM BPM and Oracle - another interesting problem

Earlier this week, I was working with a client to grow their BPM development environment from two to four nodes, meaning that the Deployment Environment effectively doubled in size.

We achieved this by editing the template ( Advanced-PS-ThreeClusters-Oracle.properties ) and adding the additional two nodes, each hosting three new cluster members, and then using the BPMConfig.sh script to update the Deployment Environment as follows: -

./BPMConfig.sh -create -de Advanced-PS-ThreeClusters-Oracle.properties

Having done this, we started up the DE, and validated the changes by hitting the Process Portal, Process Admin and BPC UIs, and also stopping the old nodes ( 1 and 2 ) and ensuring that the service carried on running on nodes 3 and 4.

Apart from briefly forgetting to regenerate/propagate the WebSphere Plugin configuration and then restart IBM HTTP Server, all was well.

However, when we started the core business application ( which is an Enterprise Archive comprising SCA modules with mediations and BPEL flows ), we saw this in the SystemOut.log for all of the AppCluster members, both the original AND the new ones: -

java.lang.NoClassDefFoundError: oracle.xdb.XMLType
Caused by: java.lang.ClassNotFoundException: oracle.xdb.XMLType
       at java.net.URLClassLoader.findClass(URLClassLoader.java:602)
       at com.ibm.ws.bootstrap.ExtClassLoader.findClass(ExtClassLoader.java:243)
       at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:777)
       at java.lang.ClassLoader.loadClass(ClassLoader.java:754)
       at com.ibm.ws.bootstrap.ExtClassLoader.loadClass(ExtClassLoader.java:134)
       at java.lang.ClassLoader.loadClass(ClassLoader.java:731)

Given that this had worked before the change and given that we know what the change was, this was somewhat weird.

This is covered here: -


*BUT* nothing has changed … apart from what has changed.

We dug further through the logs, but to no avail.

We then started to dig into the WebSphere Classloader, using this as source: -


and this: -


specifically by tracing the class loader : -

Enable Java™ Virtual Machine (JVM) classloader traces through the Admin console:

• Select Servers, choose Application servers and select the server you want to configure.

• In the Server Infrastructure section, open Java and Process Management and select Process Definition.

• Under Additional Properties, select Java Virtual Machine.

• Check the Verbose class loading checkbox.

• Add the following string to the Generic JVM arguments field:

-Dws.ext.debug=true  -Dws.osgi.debug

• Click OK.

Once we did this, and looked at the trace, we could see that, whilst the Oracle JDBC driver ( ojdbc7.jar ) *WAS* being loaded ( and we could see that, partly because BPM was coming up and working, and partly because the JDBC Test Connection function worked for all of the databases, both BPM and application ), the requisite Oracle XML tool ( xdb6.jar ) was NOT being loaded.

However, when we looked at the configuration of the JDBC driver that was being used by the application-specific data source, it looked to be in order, similar to this: -


( although we're using the ojdbc7.jar rather than ojdbc6.jar as per the screenshot above ).

When we checked the underlying Linux file-system, we could see that ojdbc7.jar was present : -

ls -al /opt/ibm/WebSphere/AppServer/jdbcdrivers/Oracle/

total 6632
drwxr-xr-x 2 wasadmin wasadmins      40 Jul 10 19:59 .
drwxr-xr-x 5 wasadmin wasadmins      45 Jul 10 19:59 ..
-rw-r--r-- 1 wasadmin wasadmins 3389454 Jul 10 19:59 ojdbc6.jar
-rw-r--r-- 1 wasadmin wasadmins 3397734 Jul 10 19:59 ojdbc7.jar

*BUT* that the xdb6.jar was NOT in the same location.

When we dug further, we could see that xdb6.jar was here: -

/opt/ibm/WebSphere/AppServer/lib/ext

along with ojdbc7.jar and xmlparserv2.jar.

which is interesting.

This then led the client to drill into the WebSphere Variable ORACLE_JDBC_DRIVER_PATH which is referenced at the JDBC driver level.

This was set to: -

${WAS_INSTALL_ROOT}/jdbcdrivers/Oracle

which is, in effect, this path: -

/opt/ibm/WebSphere/AppServer/jdbcdrivers/Oracle/

Looking at another environment, it was clear that, at some point in the past, this variable had been altered to point here: -

/opt/ibm/WebSphere/AppServer/lib/ext

rather than here; -

/opt/ibm/WebSphere/AppServer/jdbcdrivers/Oracle/

But, and here's the interesting part, the BPMConfig.sh script must've reset the variable back to the IBM-supplied default.

So, the moral of the story is two-fold; try and avoid altering IBM-provided variables AND learn how to debug the class loader in WAS :-)

Introducing IBM Business Process Manager 8.6 and there's more

IBM Business Process Manager V8.6 delivers product simplifications to utilize its full power more easily and provide greater flexibility and new capabilities


Wednesday, 23 August 2017

IBM HTTP Server and HTTP Strict Transport Security (HSTS)

I was asked about this earlier today.

IBM HTTP Server (IHS), being based upon Apache, can do most whatever Apache itself can do.

This means that HTTP Strict Transport Security (HSTS) *can* be enabled in IHS.

I followed this blog post: -


and I can see the header being set: -

 


This is what I did in httpd.conf : -

Disabled HTTP

#Listen 8080

Enabled SSL and enforced TLS 1.2

LoadModule ibm_ssl_module modules/mod_ibm_ssl.so
Listen 8443
<VirtualHost *:8443>
        SSLProtocolEnable TLSv12
        SSLProtocolDisable SSLv2 SSLv3 TLSv10 TLSv11
        SSLEnable
</VirtualHost>
KeyFile /opt/ibm/HTTPServer/ODM/ssl/keystore.kdb
SSLDisable

Added in mod_headers

LoadModule headers_module modules/mod_headers.so

Added in the appropriate HSTS header - using two years as an example expiration period ( 2 * 365 days * 24 hours * 60 minutes * 60 seconds )

        Header always set Strict-Transport-Security "max-age=63072000; includeSubdomains;"

giving me this: -

LoadModule headers_module modules/mod_headers.so
LoadModule ibm_ssl_module modules/mod_ibm_ssl.so
Listen 8443
<VirtualHost *:8443>
        Header always set Strict-Transport-Security "max-age=63072000; includeSubdomains;"
        SSLProtocolEnable TLSv12
        SSLProtocolDisable SSLv2 SSLv3 TLSv10 TLSv11
        SSLEnable
</VirtualHost>
KeyFile /opt/ibm/HTTPServer/ODM/ssl/keystore.kdb
SSLDisable

with the configuration being validated using wget : -

wget --no-check-certificate https://odm.uk.ibm.com:8443/index.html --server-response

--2017-08-23 10:48:31--  https://odm.uk.ibm.com:8443/index.html
Resolving odm.uk.ibm.com (odm.uk.ibm.com)... fe80::20c:29ff:fe9a:9e56, 192.168.153.133
Connecting to odm.uk.ibm.com (odm.uk.ibm.com)|fe80::20c:29ff:fe9a:9e56|:8443... connected.
WARNING: cannot verify odm.uk.ibm.com's certificate, issued by '/CN=odm.uk.ibm.com':
  Self-signed certificate encountered.
HTTP request sent, awaiting response... 
  HTTP/1.1 200 OK
  Date: Wed, 23 Aug 2017 09:48:31 GMT
  Strict-Transport-Security: max-age=63072000; includeSubdomains;
  Last-Modified: Mon, 31 Oct 2016 10:10:36 GMT
  ETag: "da5-540266b425f00"
  Accept-Ranges: bytes
  Content-Length: 3493
  Keep-Alive: timeout=10, max=100
  Connection: Keep-Alive
  Content-Type: text/html
Length: 3493 (3.4K) [text/html]
Saving to: 'index.html'

100%[================================================================================================================================================================>] 3,493       --.-K/s   in 0s      

2017-08-23 10:48:31 (357 MB/s) - 'index.html'saved [3493/3493]

and apachectl : -

/opt/ibm/HTTPServer/bin/apachectl -DDUMP_SSL_CONFIG -f /opt/ibm/HTTPServer/ODM/conf/httpd.conf

 

SSL configuration:
Default server
Server name: odm.uk.ibm.com:8080
SSL enabled: NO

SSL server defined at: /opt/ibm/HTTPServer/ODM/conf/httpd.conf:852
Server name: odm.uk.ibm.com:8443
SSL enabled: YES
FIPS enabled: 0
Keyfile: /opt/ibm/HTTPServer/ODM/ssl/keystore.kdb
Protocols enabled: TLSv12
Ciphers for SSLV2: (protocol disabled)
Ciphers for SSLV3: (protocol disabled)
Ciphers for TLSv10: (protocol disabled)
Ciphers for TLSv11: (protocol disabled)
Ciphers for TLSv12: (defaults) TLS_RSA_WITH_AES_128_GCM_SHA256(9C),TLS_RSA_WITH_AES_256_GCM_SHA384(9D),TLS_RSA_WITH_AES_128_CBC_SHA256(3C),TLS_RSA_WITH_AES_256_CBC_SHA256(3D),TLS_RSA_WITH_AES_128_CBC_SHA(2F),TLS_RSA_WITH_AES_256_CBC_SHA(35b),SSL_RSA_WITH_3DES_EDE_CBC_SHA(3A)

Syntax OK

Hope this helps.

Kubernetes and IBM Bluemix: How to deploy, manage, and secure your container-based workloads

A rather useful series of blog posts: -


which dovetails with my personal interests right now, given that I've just successfully brought up: -

(a) a containerised instance of IBM DB2 Express using Docker on my Mac
(b) the same on IBM Containers on IBM Bluemix
(c) the same but on Kubernetes on IBM Bluemix ( leveraging Clusters, Deployments, Pods and Nodes ).

My next challenge is to add WebSphere Liberty Profile into my Kubernetes cluster and see whether I can get a Java servlet talking via JDBC to DB2, which shouldn't be too hard #FamousLastWords

Meantime, enjoy the articles ...

Tuesday, 22 August 2017

Doh, IBM DB2 on the IBM Container Service on IBM Bluemix, remember persistence :-)

Having got DB2 running on the IBM Container Service on IBM Bluemix: -


I was a little perturbed when my Java code failed to work, when connecting from my Mac to the DB2 container: -

java -cp db2jcc4.jar:. JdbcTestDB2 54.23.128.93 50000 SAMPLE db2inst1 Qp455w0rd!

com.ibm.db2.jcc.am.SqlException: DB2 SQL Error: SQLCODE=-1031, SQLSTATE=58031, SQLERRMC=null, DRIVER=4.23.42

I checked the SQL code: -

db2 ? sql1031

SQL1031N  The database directory cannot be found on the indicated file
      system.


Explanation: 

The system database directory or local database directory could not be
found. A database has not been created or it was not cataloged
correctly.

The command cannot be processed.

User response: 

Verify that the database is created with the correct path specification.
The Catalog Database command has a path parameter which specifies the
directory where the database resides.

sqlcode: -1031

sqlstate: 58031


which made me think: -

db2 list db directory

SQL1031N  The database directory cannot be found on the indicated file system. 
SQLSTATE=58031


and then it hit me ….

I'd previously dropped the running container: -

cf ic stop db2

removed it: -

cf ic rm db2

and started a new one: -

cf ic run --name db2 -p 50000:50000 -e DB2INST1_PASSWORD=Qp455w0rd! -e LICENSE=accept -d registry.eu-gb.bluemix.net/david_hay/db2expressc:pamfixed db2start

and validated it: -

cf ic ps -a

CONTAINER ID        IMAGE                                                       COMMAND             CREATED             STATUS              PORTS               NAMES
3cf533af-c95        registry.eu-gb.bluemix.net/david_hay/db2expressc:pamfixed   "db2start "         50 seconds ago      Running             50000/tcp           db2


and requested an IP address: -

cf ic ip request

OK
IP address "54.23.128.93" was obtained.


and bound the IP address to the container: -

cf ic ip bind 54.23.128.93 db2

OK
The IP address was bound successfully.


So, of course, I no longer had any databases :-)

To mitigate this, I started a shell to the running container: -

cf ic exec -i -t db2 /bin/bash

switched to the db2inst1 user: -

su - db2inst1

and created the SAMPLE database: -

db2sampl

  Creating database "SAMPLE"...
  Connecting to database "SAMPLE"...
  Creating tables and data in schema "DB2INST1"...
  Creating tables with XML columns and XML data in schema "DB2INST1"...

  'db2sampl' processing complete.


and validated that all was well: -

db2 connect to SAMPLE

   Database Connection Information

 Database server        = DB2/LINUXX8664 10.5.5
 SQL authorization ID   = DB2INST1
 Local database alias   = SAMPLE


db2 select "EMPNO,FIRSTNME,LASTNAME from DB2INST1.EMPLOYEE"

EMPNO  FIRSTNME     LASTNAME       
------ ------------ ---------------
000010 CHRISTINE    HAAS           
000020 MICHAEL      THOMPSON       
000030 SALLY        KWAN           
000050 JOHN         GEYER          
000060 IRVING       STERN          
000070 EVA          PULASKI        
000090 EILEEN       HENDERSON      
000100 THEODORE     SPENSER        
000110 VINCENZO     LUCCHESSI      
000120 SEAN         O'CONNELL      
000130 DELORES      QUINTANA       
000140 HEATHER      NICHOLLS       
000150 BRUCE        ADAMSON        
000160 ELIZABETH    PIANKA         
000170 MASATOSHI    YOSHIMURA      
000180 MARILYN      SCOUTTEN       
000190 JAMES        WALKER         
000200 DAVID        BROWN          
000210 WILLIAM      JONES          
000220 JENNIFER     LUTZ           
000230 JAMES        JEFFERSON      
000240 SALVATORE    MARINO         
000250 DANIEL       SMITH          
000260 SYBIL        JOHNSON        
000270 MARIA        PEREZ          
000280 ETHEL        SCHNEIDER      
000290 JOHN         PARKER         
000300 PHILIP       SMITH          
000310 MAUDE        SETRIGHT       
000320 RAMLAL       MEHTA          
000330 WING         LEE            
000340 JASON        GOUNOT         
200010 DIAN         HEMMINGER      
200120 GREG         ORLANDO        
200140 KIM          NATZ           
200170 KIYOSHI      YAMAMOTO       
200220 REBA         JOHN           
200240 ROBERT       MONTEVERDE     
200280 EILEEN       SCHWARTZ       
200310 MICHELLE     SPRINGER       
200330 HELENA       WONG           
200340 ROY          ALONZO         

  42 record(s) selected.

I then re-ran my Java code: -

java -cp db2jcc4.jar:. JdbcTestDB2 54.23.128.93 50000 SAMPLE db2inst1 Qp455w0rd!

000010 CHRISTINE HAAS
000020 MICHAEL THOMPSON
000030 SALLY KWAN
000050 JOHN GEYER
000060 IRVING STERN
000070 EVA PULASKI
000090 EILEEN HENDERSON
000100 THEODORE SPENSER
000110 VINCENZO LUCCHESSI
000120 SEAN O'CONNELL
000130 DELORES QUINTANA
000140 HEATHER NICHOLLS
000150 BRUCE ADAMSON
000160 ELIZABETH PIANKA
000170 MASATOSHI YOSHIMURA
000180 MARILYN SCOUTTEN
000190 JAMES WALKER
000200 DAVID BROWN
000210 WILLIAM JONES
000220 JENNIFER LUTZ
000230 JAMES JEFFERSON
000240 SALVATORE MARINO
000250 DANIEL SMITH
000260 SYBIL JOHNSON
000270 MARIA PEREZ
000280 ETHEL SCHNEIDER
000290 JOHN PARKER
000300 PHILIP SMITH
000310 MAUDE SETRIGHT
000320 RAMLAL MEHTA
000330 WING LEE
000340 JASON GOUNOT
200010 DIAN HEMMINGER
200120 GREG ORLANDO
200140 KIM NATZ
200170 KIYOSHI YAMAMOTO
200220 REBA JOHN
200240 ROBERT MONTEVERDE
200280 EILEEN SCHWARTZ
200310 MICHELLE SPRINGER
200330 HELENA WONG
200340 ROY ALONZO


So that's all good then :-)

The moral of the story - when you drop and recreate a container, don't assume that the "local" storage is still there; next time around, I'll mount some external storage to my container ….