How to Optimize Your Watson Chat Bot Application in Production

You’ve gone live into Production with Watson, great! So, the question is, what do you do now? How do you measure your deployment? How do you improve the deployment? I’d like to talk about some best practices and recommendations on how to optimize your Watson deployment once its in production. These best practices are being based on my experiences with these Watson solutions: WEA (Watson Engagement Advisor), Dialog/NLC (Natural Language Classifier) and the Watson Conversation service available on IBM Bluemix. However, these principles can be applied to other Watson services as well.

Qualitative Analysis

I just want to be clear that the approach I am speaking of below is one measurement that is specific to improving and optimizing the question-to-answer level quality (eg fine-tuning intents, see which questions are doing well/not well). To determine the true value of any cognitive chat solution, you need to look at the overall conversation level to measure if the chat provided value (To keep this focused, short and sweet, I will not go into the conversation level detail here).

One of the keys to optimizing Watson is consistent qualitative analysis (in this case, at the question-to-answer level). This is validation of the quality of Watson’s performance on a regular and consistent basis. You can setup the right frequency and numbers that work for your organization based on maturity of the deployment, deployment size, volume of activity (eg questions being asked) and resources available, but I will provide an example.

For your first time doing this, you can pull 500 questions from your production deployment’s conversation logs (which you would do every 2 weeks) and analyze the questions asked along with the responses provided. You should create a rating scale on how you rate the responses based on correctness. For example:

4 – Perfect answer
3 – Acceptable answer
2 – Not an acceptable answer
1 – Completely unacceptable or ridiculous answer

Your goal is to assess what answers were acceptable/correct and which answers were not, which leads to what questions require attention. Once you’ve rated all 500 of those questions/responses, add up the number of 3s and 4s as “correct” and add up the 1s and 2s as “not correct” and that percentage becomes your baseline number for correctness. Keep this baseline number handy as you will need it. Now, you’re ready to take some action.

Take Action

I highly suggest that you look for trends of question intents where Watson is not performing well. I would put much less emphasis on “one-off” types of questions that Watson did not respond well to (unless it’s a critical question to the business or a liability topic). You want to focus on what the majority of your end users are asking to have the largest impact, so again, look for trends.

Once you identify a trend, you can begin to look at ways to enhance the question intent. There are many corrective actions that you could take but it all depends on your analysis of the intent. Maybe you need to add more variations to the intent to expand Watson’s understanding of how this question could be asked or phrased? Maybe you need to adjust your intent groups so that Watson can better delineate between them? I have another blog post that talks about Best Practices for Training Watson on Intents that I suggest you read to help you.

So up until now, you’ve taken a question sampling, rated that sampling on correctness, analyzed for trends and taken some corrective action. So what is next?

Iterate!

Once it’s been two weeks since your initial sampling of questions, pull another sample of 500 questions. Another important thing here is to pull the same number of questions and analyze/rate them the same exact way. You also want to have the same person(s) doing this work. This is to ensure consistency and continuity across each iterative cycle.

Now once you’ve run through the cycle again (and each time thereafter), there are several key things you should be looking at:

1) Compare the correctness rating with the previous cycle. Are you doing better, worse or the same?
Note: You are looking for incremental improvement, so a consistent incline of even half a percent is positive progress.

2) Compare the trends of intents you identified that are not doing well with the previous cycle to see if you still have room for improvement on the same intent. Are you not performing well in the same intent(s) over and over? If so, then you need to seek Expert Advice to look into what is going wrong.

Evaluate

There is no “one size fits all” guideline as to how long one should iterate on the qualitative analysis cycles to improve their Watson deployment. There are many factors that vary from deployment to deployment such as:

  • Deployment size – The number of intents/topics covered in the deployment
  • If new intents/topics are added to the deployment
  • If requirements change
  • If new user groups/organizations are provided access to the deployment

As you evaluate Watson’s performance over time, you should compare your rating numbers and observe the trend. Eventually, you will hit the point of diminishing returns (unless you continue to add new intents/topics into the solution). You will need to make the judgement call of when you hit this point.

Feedback

One other helpful aspect you could analyze is end user feedback. I have often seen many chat applications implement a thumbs up/thumbs down rating per each content response provided by Watson. This feedback could be misleading as the end user could have given a response a thumbs down, not because it was not the right answer, but because they perhaps just did not like the answer. I highly suggest looking for trends of questions with answers rated as thumbs down. If there are a group of people all rating a specific question intent’s response as thumbs down, then I suggest having a closer look to see what corrective action may be needed. Again, focus on trends, versus a “one-off”.

The End

Optimizing Watson is not always an easy task, but following this qualitative analysis prescription consistently as described should help provide some guidance in making the task a bit easier.

Best Practices & Lessons Learned for Training Watson on Intents using Conversation Service (or Natural Language Classifier)

How to properly train Watson to understand intents is a very popular topic. This can be done using the new Watson Conversation Service (or the Natural Language Classifier service) on the IBM Bluemix platform. I have been asked on a weekly, and sometimes daily, basis on how this is best done. In an effort to help share my experience and expertise in working with Watson over the last few years, I will provide some guidance for beginning and improving your training of intents.

Reality

The fact is that there are several general best practices that one could follow but every Watson deployment will be unique and require some unique tweaking. Some practices you may not need to follow, some you may need more of and some things you will do different. This is cognitive and this is reality. You will need to think innovatively of how you can take these core practices and leverage them in your deployment.

Best Practices

Use representative end user questions – these are questions that are captured from end users themselves, in their own words and terminology. This may be a bit more challenging for some organizations to obtain. Some organizations have an existing web chat or online support service that they can scrub representative questions from to use as a starting point. For those organizations that do not have this type of data available, there are other processes that can be implemented to obtain end user questions, such as polling real end users through an existing system or through a new one.

Use atleast 10 variations to train each intent – the more diverse the representative question variations you obtain are, the more thoroughly you can train your intent. For optimal results, I would target to use more than 10 variations but this depends on how well your testing goes (a later topic). I’m not saying you need 300 variations per intent, which seems overboard, but 10 is the minimum as a general guideline in my experience.

This will also depend on how many intents you have in your deployment. In cases where you have a low number of intents, such as 12 intents total, you may not need to provide more than 10 variations to achieve high confidence if the intents are very distinct.

Variations – this one has been a huge topic when training Watson. Using Watson Engagement Advisor in the days before the Conversation service/NLC, the guidance that I had followed successfully, was always to use representative questions as variations. I always followed that approach for that technology and it was proven successful.

The same approach should be followed for the Conversation service/NLC. The best practice is to collect representative questions from the actual end users, identify primary intents and use the remaining like-intent questions as variations to train Watson.

I’ve been asked “Can I make up some of my own variations to add or do those all have to come from the actual end users?” The usual guidance is that we should not be self-creating variations. In working with the Conversation service/NLC, I’ve found that there may be, at times, an exception to this case. In some rare circumstances, I have not been able to get Watson to understand an intent with a high confidence level, even after adding many representative question variations. Folks, this is cognitive and there very well may be clear justification somewhere deep in Watson’s algorithmic mind, but it may not be obvious to the human eye/mind. There have been cases where I was left no choice but to provide a self-generated variation to help Watson learn better.

I want to be crystal clear that this approach should be used as a last resort and handled with care. I do not recommend starting with this approach or overusing this approach. I stand by the approach that everyone should be using representative end user questions as their foundation for intents and variations. Intents should never be fabricated, ever. Variations can be supplemented with self-created variations on a very rare and careful basis. If you self-create all your intents, you will likely experience sub-par performance for trying to fabricate the end user speak. If you find that you need to constantly add in self-created variations to properly train, then you need to revisit your collected representative questions as there is something that is really wrong.

Structuring Intents – There are two main methods by which you can structure your intents. It is very popular and common that you will use a combination of these in your deployment.

One way is to group intents that do not contain various different entities embedded in each intent. Here is an example of some questions of an intent designed to ask about general team performance:

Can you tell me how my team is performing?
How is my team doing?
Are my team members performing well?

The other way method to structure your intents is to group questions which include multiple entities embedded in them. This is when you have one high level intent, but little sub intents within them. You would group them all together as one intent to give Watson a stronger confidence level of the high level intent, and then in Conversation/Dialog, you would use entity extraction to pull out what the sub intent is. Here is an example:

Can you tell me how my team is performing?
Can you show me performance across all teams?
How is the team performing in the US?

As you can see, all three of these questions are asking about team performance but focusing on 3 different aspects of it. Depending on your case, you could train Watson on all of these as one intent called “Team_Performance” and then use entity extraction to examine the user utterance for the sub intents (eg “across all teams” or “in the US”).

*Technically, there is another possibility where you could identify sub-intents by using multiple classifiers/intent services but that would require additional effort and configuration at the orchestration layer that I will not go into here.

Iterative Training & Testing Cycles – This is key to the Watson learning process. The sooner you can put your Watson deployment in front of real end users, the higher quality your deployment will be when you go live. Again, in some organizations, this may be easier to achieve, such as if your end users are internal employees. However, this may be more challenging for organizations where their end users are external customers, but still achievable.

I would use the same number of questions per test session and craft the sessions around the same topics (to ensure the test questions are targeted against the use case intents). Once you’ve achieved satisfactory performance, you can move on to other topics. After your first test session, capture your results that include the percentage of questions that returned the proper intent. This will become your baseline. Once you’ve got your baseline, thoroughly review the results and look for trends of questions that require improvement. You should start with the most poorly performing intents, whether they are existing intents or new intents. Your approach of calibration could be refining intent groups, removing questions, adding questions, splitting intent groups, or adding new intent groups.

Conclusion

I hope this helps provide you additional guidance to model, train, test and calibrate your intents using the Watson Conversation service (or the Natural Language Classifier service). I will continue to update this blog post to share any new information that I may learn as we continue on through the cognitive journey.

Slack – How it adds value to your workflow!

Earlier this year, I started to use Slack as a collaboration platform to attempt to work more efficiently with my project teams. Many of my colleagues and friends have approached me and said:

 

“What is so great about Slack?”
“Why do I need it if I use Sametime [Connect]?”
“Here we go with yet another tool I have to use.”

 

My answer is this. If you want a comprehensive list of all the bells and whistles of Slack, you can read about them on Slack.com. If you want to know the tremendous value my team and I have realized and how it helped us truly work more efficiently, saved time and increased productivity, then read these key points below..

 

Key Value & Benefits

 

  • Less Meetings – Reduction of ~50% in conducting formal meetings
  • Less Email – I saw a reduction of ~80% in email communication in my project team
  • Increased Productivity – We not only delivered on the defined activities of our project, we overachieved by producing additional wow factors and over delivered on most of the defined activities
  • Increased Time Savings – Did you see my comment above about less emails and less meetings? Great, then you get this!
How I Realized Tremendous Value By Using Slack

 

  1.  Can be logged on from multiple devices. I was always logged onto Slack. I downloaded the app on my MacBook and iPhone. Regardless of when I was not in front of my MacBook, I was always connected with my project team. This helped us work faster. You cannot be logged on to multiple devices using ST Connect in my organization. I highly suggest anyone using Slack downloads the respective app for your device(s). 
  1. Chat History at your fingertips. Slack maintains all the chat history within each chat, whether a direct message or a channel. This made it very easy to recall prior conversations, statements and questions.
  1. Image sharing made easy – I love this feature. When you want to share a screen capture or image with someone, you just paste it right into the chat and Slack automatically creates a tile/preview version of that image. You can also add comments to accompany the snippet. This is very useful as it does not pollute the chat interface like typical chat solutions do.

slack_image

  1. Collaboration with team members outside my organization – I’ve had team members on projects that are not part of my direct organization or business partners. Thus I had no platform to communicate with them in one place. Slack provided us a forum to all come together and collaborate on the fly, regardless of which organization you belong to. No extensive paperwork filed or long turnaround times, with Slack it’s all made easy.
  1. Tagging users – If your working in your project team’s channel and need their immediate attention, you can tag the user (eg @mjnehme) and this will provide an additional alert to that user. If you need to alert the entire team on the channel, you can do so (eg @channel), but that should only be used when really needed. In this example below, I tagged myself.

Snip20160825_111

  1. Speed – If I ever needed anyone, I just “slacked” them. Everyone was always online, so it was easy to get an answer to something quickly that may be gating you. Its literally a virtual room. 
My final piece of advice is to the BE LOGGED ON! For Slack to be effective and provide the value I have highlighted, your entire team just needs to be logged on at all times (for atleast your planned “work schedule”).

 

Would Facebook, Twitter or Instagram be as successful if they did not have daily active users? Not likely. Its the same principle here. I found huge productivity gains in my projects but this only works if the team is all on board.

 

Hope this helps provide value to all your projects! #SlackIt

How to configure Cognos for Microsoft Office

This is a Cognos plugin tool that is used to integrate your IBM Cognos BI, IBM Rational Insight or RRDI environment with Microsoft Office so that you can view Cognos reports through Office software. The tool has also been called “Cognos Office” or “Cognos Go-Office” in the past. As of this post, it is called “Cognos for Microsoft Office”.

I have found the best use for this tool is using it with a Powerpoint presentation. Here is an example. Say an employee has a weekly meeting where they present a specific set of slides. The structure of their slides are the same but the data in the reports being shown (which come from Cognos) vary from week to week. So each week, this employee must manually generate the reports via copying and pasting screen shots into the Powerpoint presentation. Its not the most enjoyable task.

However, if this employee installs the “Cognos for Microsoft Office” plugin, then every week, they would simply launch their existing Powerpoint slide deck and click on a “Refresh” button and this will automatically update their existing reports to show the most current data. That is all, in one simple click.

This plugin is a local install, so each user that wants to use it must install it on their local machine, where their Microsoft Office software resides. It is a pretty straight-forward installation and configuration process. Once you have installed the software, here is how to configure it on your local machine. (Please note that I am using IBM Cognos 8 in this example).

1) After installation of the Cognos for Microsoft Office plugin, launch Microsoft Powerpoint and you will find a new tab named “IBM Cognos 8”. Click on this tab.

2) Now, in the “Application” section, click on the IBM Cognos 8 button, then select “Options”

3) Here, specify your Report Server that Office will communicate with. Enter the “System Gateway URI” and give it a “Friendly name”.

4) Click “Test Connection” to verify that your Office software can communicate with your Report Server. If your connection is successful, you should see this message:

5) Now, click “Add” then “Ok”

6) From the toolbar, click “Logon” and select the Report server you just added (in my case, its “My Insight Server”)

 

7) Login with your Report Server credentials (if applicable)

8) Now, on the right hand side of your screen, click on “IBM Cognos 8 Go! Office” (Depending on your version, you may not see this identical text)

 

9) You should now see your Report server directory structure in your Microsoft Office Powerpoint application

 

10) You can now drag and drop your Cognos reports into your Powerpoint presentation

Best Practices on Planning / Scoping a Cognos Reporting Environment

I thought I would share my expertise and experience on best practices in planning a new deployment for a reporting environment. I have had a ton of experience with many different organizations and industries, worldwide. I have seen alot of successful deployments and I have seen alot of…well…unsuccessful deployments.

While much of what I am about to discuss could be applicable to any software deployment, I will focus this specifically on a IBM Cognos BI, IBM Reporting for Development Intelligence and IBM Rational Insight deployment. There are many different aspects to take into consideration when you are trying to scope and plan a new reporting environment. Here is my advice on what you should do.

Start small and grow big!

You cannot address all of your needs or solve all of your problems in the first phase of your deployment. It is a common mistake to onboard too many users too quickly or over scope the initial deployment. This can lead to many issues including performance problems, server stability, user frustration, an abundance of problem tickets and even canceling the entire project effort.

You should take an incremental approach to your deployment and you will should do this in an iterative fashion. First, you need to scope a type of topology based on your current, and what you believe to be, your future needs (eg clustered , non-clustered, IHS server/Cognos Gateway or a standard distributed environment). To do this, you need to review your overall business goals, translate those to operational objectives and then break those down into individual tasks.

Here are some questions you should address when attempting to plan for a new Reporting environment:

  • What are my expected number of users in the first 3 months, 6 months, 1 year?
  • What types of reports will these users be running (live, trend, saved)?
  • Will the reports be run individually or through dashboards or schedule and emailed to stakeholders?
  • Will my users be accessing the Report server directly or through another interface (eg Rational Team Concert dashboard)?

Once you have understood these items and identified your topology type based on this, you now have the foundation for your initial deployment (note: it is possible to change your deployment topology after the fact, such as going from a vertical to horizontal cluster). Remember, start small and grow big.

The next thing you need to do is identify a small set of users (eg ~50) who will access the deployment as early adopters. You also need to identify a small set of reports (eg 5-10) to develop and deploy into production for use. Again, do not try to do too much too fast. History and experience has shown me that this approach of trying to do too much too fast has a significantly lower percentage of success.

Once you go live with your initial deployment, you need to closely monitor it and look at these key elements:

Performance

  • How are my servers performing?
  • How are my cpu and memory utilization?
  • Are my servers crashing?
  • How is the network behaving?

Usage model / User activity

  • Are my users consuming the solution as expected?
  • Are they logging into the Report server portal directly?
  • Are they accessing reports through other means such as a Rational Team Concert dashboard widgets?
  • Are they scheduling reports and having them emailed to stakeholders?
  • Are they running individual reports or viewing a dashboard of reports? How many reports (and what type) are being run in each dashboard?

User feedback

  • What are my users saying? What is their experience with the deployment?
  • Am I getting alot of problem tickets or emails about errors?
  • Are they getting any value from the deployment?

I highly recommend that you set milestones for your deployment. Here is an example of a high level roadmap. This is just an example! You can scale this however you deem appropriate so long as you start small and expand incrementally.

As you progress through milestones, you should be making adjustments based on your observations of what I mentioned that you need to monitor. You may find that you need double the amount of servers you had scoped. In this case you would either need to expand your topology quicker or onboard users at a slower rate. You may also find that you have more than enough of power within your server environment. In this case, you could either scale back a few servers, downgrade some of the machine specs or onboard more users at a faster rate.

The key to success is to start small and grow big!

Links to my IBM Innovate Conference presentations

Here are links where one can view or download my IBM Innovate presentations over the last five years.

Innovate 2010: Best Practices and Lessons Learned on Our IBM Rational Insight Deployment

Innovate 2011: The IBM Rational Insight Reporting Solution

Innovate 2012: Deploying Rational Insight into a Heterogeneous Environment

Innovate 2013: Improving Predictability and Efficiency with Kanban metrics using IBM Rational Insight

Innovate 2014: Unleash Your Metrics Outside the Box: Customizing Your IBM Rational Insight Deployment (workshop)

Configuring Rational Insight to extract data from Rational DOORS

In this post, I will show you how to configure IBM Rational Insight to extract data from IBM Rational DOORS using the RIF integration method. In this scenario, I am using Rational Insight v1.0.1.1 & Rational DOORS v9.3.0.4. This is based on documentation I created on September 9th, 2011 and it is still valid as of this post. However, if you are using Rational DOORs v9.5.1 or higher, you should leverage the more optimal REST interface integration provided by the IBM Rational DOORS team here.

The way to achieve this integration is to point Insight to a DOORS RIF file (which is exported from DOORS) that contains the necessary data to be reported on. Before I begin, let’s review some key things to keep in mind to communicate to your DOORS Developer whom will be defining the content of the DOORS RIF file for you to point to from Insight.

  • Ensure that all of the necessary fields that you are looking to capture in Insight are available in the DOORS view in the RIF file.
  • Only String and Integer field types of DOORS are consumable in the RIF format. If you’ve got some fields stored in a Text type field, for example, that you require, then you must create another new field of type String and write a DXL script to populate that “Text” field’s data in the new field. It will then be consumable in the RIF format.

Now, let’s start with the integration..

Configuring the XDC file – There are two different methods when pointing to a RIF file

Method 1 – Pointing to a static RIF file

1) Let’s say that the DOORS developer has created and exported a RIF file and placed it on your C drive (Windows OS) of your Insight developer machine. In this scenario, here is how you would configure the XML Data Configuration tool (XDC) to point to this RIF file, called myRIF3.xml.

2) Launch the XDC tool and open the doors.xdc configuration file (by default, located here: (Insight_install_directory)\etl\configs\Configurations)

3) Traverse through the Resources directory until you reach the Resource Group level. Double-click on the DOORS resource group and ensure the URL Type is set to RIF, the URL is pointing to the correct directory of the RIF file on your local machine (in this case, the C drive), and Authentication type is set to None. (Note: You can optionally test the connection at this point by click on Test Connection but this does not mean that you are successfully integrated with the RIF file just yet).

4) Next, double-click on the Resource named doors and ensure that the relative path references the name of the RIF file

5) Now, you must verify that you can review the contents of that RIF file from Insight. Double-click on one of the Data Mapping Tables, let’s select Header. Then click on the Columns tab at the bottom left of the workspace window

6) Now click on the Modify link to view the XML schema of the RIF file against your data mapping table

7) If you have completed all of the previous steps successfully, then you should now see the XML schema of the RIF file under the section labeled XML Schema. If that section comes up blank, then there is either an issue with the source RIF file or you may have missed one step above.

 

Method 2 – Pointing to the RIF Exporter Service

The other method that could be used to extract data from DOORS into Insight would be to point to the RIF Exporter Service. The RIF Exporter Service is an IBM Services asset that automates the generation of RIF files. For more information, contact your local sales representative.

Assuming you have the RIF Exporter Service deployed, here is how you would configure Insight to leverage it.

1) Launch the XDC tool and doors.xdc file (as shown in the steps above). Traverse through the Resources directory until you reach the Resource Group level. Double-click on the DOORS resource group and ensure the URL Type is set to RIF.

The URL must be pointing to the RIF file located at DOORS RIF Exporter service. Sample syntax is: http:/DoorsServices/DOORS/InsightP?rifdef=. Here is my example: http://localhost:9080/DoorsServices/DOORS/InsightP?rifdef=myRif3

The Authentication type should be set to Basic and you must enter credentials that have access to the source DOORs deployment. You may now click on Test Connection to verify that you can communicate with that file.

2) Next, click on the Resource doors and verify that the Relative path box is empty. You have already specified the file name at the Resource Group level, which is used as part of the URL for validation.

3) Now, double-click on the Data Mapping Table named SpecObject (which provides the basic data around DOORS objects.

4) Click on the Modify link at the top right of the workspace

5) If you now can view the XML schema on the left without any error messages then everything that is being mapped in your Relational table columns does exist in the RIF file. You can move on to Step 2.

6) However, if you receive an error message like this, and see fields highlighted in red, then this means that there is something that exists in your mappings in Insight that does not exist in the RIF file. You will need to verify the contents of what is available in the RIF file with your DOORS administrator.

7) In this example, you will modify the XDC mappings to capture only the fields that they would like to extract from DOORS. If they go back to the Data Mapping Table view (in Columns tab), select the columns that you do not currently want to capture, and click on the Unload link at the top right.

The view will now look like this:

8) Click the Save button. Also, make sure to save any other windows you have open in the XDC tool if applicable.

Configuring the Extract Transform Load Process

1) Launch IBM Cognos Data Manager and open your current ETL catalog.

2) Navigate to the Operational Data Store (ODS) section, and expand the DOORs folder. These are the out of the box DOORS fact builds that ship with Insight.

3) In this example, you need to configure the DOORS object, so click on the DOORS_Stg fact build and then double-click on the DataSource1 node in the workspace on the right.

4) Click on the Query tab

5) Here you find the DOORS Data Mapping Tables (as defined in the XDC tool) on the left hand side, and on the right, you see the out of the box SQL Query for the SpecObject table.

You need to modify the SQL Query so that it matches what data is currently available from DOORS, based on your previous mappings.

6) The simplest way to do this is to right-click on the SpecObject node and select Add Table select statement.

This will automatically remove the existing SQL Query and replace it with a SQL Query that will extract everything in your current SpecObject mapping table. It should look like this.

7) You should validate the data here, so click on the green triangle button to return all rows. This will query the DOORS RIF file and return the data queried. You should review these results to ensure this is what you are expecting.

8) Now, click on the Result Columns tab, and then click on the Refresh button on the bottom left to refresh the result columns that you now will return. Then click on Ok.

9) At this point, you would continue to configure the ETL just as you would for any other data source (business as usual). You will need to repeat the above steps for each Data Mapping table or entity that you would like to extract into Insight.

Summary

This blog post demonstrated how to configure Rational Insight to extract data from Rational DOORS. First, I showed how you integrated Insight with the DOORS RIF file and verified that it could see the source data. Second, I modified the XDC file and exposed the DOORS data that we were interested in extracting into Insight. Finally, I configured the Data Manager ETL fact to realize the source data via the SQL Query window.

Why is my Role & Membership data Missing in the Rational CLM / RRDI or Insight Data Warehouse ?

Some folks may experience no data being returned when running reports containing role or membership data from CLM (JTS) using Cognos Report Studio with RRDI or Rational Insight. Some examples would be a report design that contains data from these sections of the framework model:

Operational Data Store > Project Area > Project Role
Operational Data Store > Project Area > Project Related members
Operational Data Store > Team Area > Team Role
Operational Data Store > Team Area > Team Related members

If you are seeing no data coming from these query subjects, then it could be because one of these reasons:

1) You are not using a JTS that is version 4.0.5 or higher (the ETLs for membership and role data for projects and teams were added in version 4.0.5)
2) Your CLM user, whose credentials you use to login to RRDI with, does not have permissions to access those project/team areas
3) Your membership/role ETLs are disabled.

I am going to provide more information on #3. By default, the ETLs for membership and role data are disabled. The reason for is likely because not every user or deployment would require reporting this data. Hence, to avoid additional ETL processing overhead, it is disabled by default. If a user wants to report on that data, then they need to enable these ETLs.

Here is how to do it.

Enabling the membership/role ETLs for RRDI in the Jazz Team Server

If you are using RRDI, then you can enable the Java ETLs by logging into the the Jazz Team Server and navigating to the Administrator section. Then, go to Server > Advanced Properties. Its easier to search for “com.ibm.team.datawarehouse.service.internal.common.CommonRemoteSnapshotService”. Here you will see the “Ignore Roles and Membership Data” property, which is set to “true” by default. Change this to “false” and save. Run a full load of the ‘Common’ job and the membership/role data will now be loaded into the data warehouse.

JTS/Admin Advanced Properties

Enabling the membership/role ETLs for Rational Insight in Cognos Data Manager

If you are using Rational Insight, then you can enable the Data Manager ETLs by logging into your Data Manager client. Then, navigate to Builds and JobStreams > Jobs > JFSJobs > JFS4.0.5Jobs > JFS_Common4.0.5. You need to enable the following fact builds:

JFS_ProjectAreaMember4.0.5
JFS_TeamAreaMember4.0.5
JFS_ProjectAreaRole4.0.5
JFS_TeamAreaRole4.0.5
JFS_ProjectAreaRoleMember4.0.5
JFS_TeamAreaRoleMember4.0.5

Data Manager - JFS_Common4.0.5

To enable each fact build, double-click on each build shown in the image above and ensure the “Exclude this node from processing” property is unchecked. Then click “Ok” and save the catalog. Run a full load of the ‘JFS_Common4.0.5’ job. The role/membership data will now be loaded into the data warehouse.

Configure fact build properties

IBM Rational Insight Enablement Links

Alot of folks ask me for information on how to do things or if certain things can be done. Many ask for some presentation collateral. Here is a list of some links to collateral that contain alot of great information about IBM Rational Insight that I have created over time that will address alot of your questions:

IBM Rational Insight Overview 2014

The IBM Rational Insight Reporting Solution

Deploying IBM Rational Insight in a Heterogeneous Environment

Improve Predictability and Efficiency with Kanban Metrics using IBM Rational Insight (presentation)

Improve Predictability and Efficiency with Kanban Metrics using Rational Insight (article version)

Integrating Rational Insight with HP Quality Center and other Third Party tools

Configure Rational Insight with an additional Jazz Team Server

Configure LDAP for Rational Insight when Integrated with Multiple Jazz Team Servers

How to implement delta loads using Rational Insight

Integrating a Microsoft Excel spreadsheet with Rational Insight (live data model)

Define Aggregate Rules for Semi-Additive Measures in Framework Manager

IBM Rational Insight – Setting up your environment for remote execution using Data Manager Network Services

Configuring Rational Insight for Rational Team Concert custom attributes (for RTC version 3.0 and earlier)

Here are some helpful IBM links:

IBM Rational Insight 1.1.1.x Infocenter

IBM Rational Insight Data Models (ETLs, Data Warehouse, Reports)

IBM Rational Insight & CLM Data Dictionaries

Insight / CLM Integration

IBM Rational Insight & IBM Rational Reporting for Development Intelligence (RRDI) – how they compare

This is another one of those situations where there can be alot of confusion between these two IBM report offerings. I did a similar comparison between IBM Rational Insight and IBM Cognos BI in a previous post IBM Rational Insight & IBM Cognos BI – How they compare. I suggest reading that post which has descriptions for several of the components that will be mentioned here. Please note that everything I mention here is accurate as of the date of this post and things are always subject to change in the future.

First, I am going to discuss what these two solutions have in common, and then I will talk about some of the key differences. Finally, I will provide some common use cases I have come across and which solution would be the most appropriate.

RRDI is an optional component when someone owns the Rational Collaborative Lifecycle Management solution or any piece of the solution. RRDI is a subset of Insight, or rather, Insight is an extension that is built on top of RRDI. The core binaries are the same. When you install the Insight report server, you will have two sets of bits to lay down, the RRDI components and the Insight extension. If you read my other post referenced above, you would already know that Insight is built using the IBM Cognos BI technology.

Both solutions contain a Report Server, which are deployed using the same bits, hence identical. The web UI navigation and experience is identical. The report authoring tools (Report Studio, Query Studio) are identical. Insight comes with some additional tools that appear in the report server which I will mention below.

Both solutions report off of Data Warehouse. The Data warehouses are almost identical for both solutions as they contain all the same core schemas. There is only one difference. The data warehouse that RRDI uses comes from the Rational CLM solution which contains an RICALM schema for certain specific CLM data points (Team Concert, Quality Manager, Requirements Composer). Insight contains an RIBA schema for some scorecard metrics. RRDI does not contain this RIBA schema. If you integrate Insight with one of the Rational CLM tools, then you will add the RICALM schema to the existing Insight data warehouse.

Now I’d like to discuss differences between the two solutions. I will only elaborate on the major differences, however I’ve provided a list here to display all of the key differences:

Differences between RRDI & Insight
*Both solutions can achieve cross-project reporting.

  1. RRDI can only report off of data that resides in the CLM data warehouse. This is typically limited to Rational Team Concert, Rational Quality Manager and Rational Requirements Composer. Please note that there are ways to load other tools’ data into the CLM data warehouse but that will not be discussed here. Insight is not limited to just CLM product data and can integrate with almost any data source in several ways (eg ODBC, REST APIs, XML).
  2. RRDI is limited to reporting on application data coming from one Jazz Team Server. Insight can report on application data from multiple Jazz Team Servers.
  3. RRDI’s framework data model and the CLM Java ETLs are non-customizable and no developer tools are provided. Insight’s framework data model and Data Manager ETLs are fully customizable and a set of developer tools are provided.

Here I have listed some common use cases that I have come across in working with clients and which reporting solution best meets those use cases.

  1. An organization has certain teams that use the IBM Rational CLM suite of tools and they also have other teams using some tools from HP and Microsoft to capture similar data. They are looking to create enterprise-level reports that aggregate data across all teams to provide a high level view to their executives.

    Appropriate Solution: Rational Insight
  2. An organization is using IBM Rational Team Concert for their change and configuration management (several CCMs with only one Jazz Team Server). They have a business need to create metrics based on their CCM data only.

    Appropriate Solution: RRDI
  3. An organization has several teams using Rational Quality Manager, Rational Team Concert and Rational Requirements Composer. Each team has their own Jazz Team Server (multiple Jazz Team Servers) and multiple CLM applications.

    Appropriate solution: Rational Insight

Everything that is Technology