Understanding Hyperion Upgrades

Any voyage or journey has a starting point and a target destination. They say it’s all about the journey and how true is that? And guess what, it’s not any different for Hyperion upgrades. However, for the sake of Hyperion let’s assume this journey is more like the Amazing Race and time & budget are limited and our aim is to complete each stage of the project correctly and on time to progress to the next stage with ultimate aim of completing the upgrade project successfully and on time & budget.

Unlike the amazing race if you do achieve this you won’t win $1 Million dollars and celebrate on TV, rather you get to celebrate at the post go-live party (yes have a party you deserve it) with the project team and dish out high 5s as much as you like. Let’s face it you love what you do, and to you, your Hyperion Upgrade project is a challenge that you won’t let defeat you.

For Hyperion upgrades the journey is as follows and the steps between start and the end goal require careful planning:

 

Upgrade Type

Let us start by understanding the various upgrade types, from there depending on the current version and strategy we can map out our journey ahead clearly. It is also essential whatever solution you choose the path you follow is completely supported, to ensure if any issues are encountered you have Oracle support at your fingertips.

• In-Place Upgrade (Maintenance Release) – This approach involves installing the latest release over the top off the existing version. Currently this is only support for 11.1.2.x versions. It is essential when using this approach to ensure the underlying platform (hardware, OS & Relational versions) is able to supported source and target versions.

• Out-Of-Place Upgrade (Upgrade) – This involves upgrading directly from the old platform (OS/Relational/Servers/Hyperion software versions) to the new platform (OS/Relational/Servers/Hyperion software versions). This is only supported for source release version 11.1.1.4.

• Staging Upgrades – This involves creating an environment purely for upgrade purposes and using this environment as the transition environment from one version to the next. Usually I recommend this is to be a compact single server environment to allow a quick upgrade actions on a single server and at the same time removing the impact on the existing environments. The process involves installing the current version of software on staging environment, upgrade staging environment to latest version and finally migrating to new environments on latest supported platform. It is essential to ensure that the underlying platform (OS & Relational versions) are supported by both source and target versions. The staging approach can be used for In-Place Upgrades or Out-Of-Place Upgrades with main purpose as describe to avoid impact on existing environments and to ensure the final environment can be on the latest possible platform for the target version.

• Hybrid Approach (In-place & Out of Place Combinations) – To upgrade some versions you may require to action both approaches. For Example, if the source environment is 11.1.1.3 and the target is 11.1.2.4. You will first need to in-place upgrade from 11.1.1.3 to 11.1.1.4, then out-place to 11.1.2.3, then in-place upgrade to 11.1.2.4. In such complex upgrades I almost always recommend the actions to be done in staging environment(s) to avoid impact the existing environments and minimise upgrade time.

• Rebuild In New Version – At times the solution is best to rebuild. The time that would be spent upgrading can be used to make changes / improvements to the existing solution. For example, if the chart of accounts was to change entirely there would potentially be design changes needed to the solution. At times after having the system for some time the business now realises exactly what they are after and now want it rebuilt a specific way. This is also an excellent way of taking advantage of new features/functionality and building the tool with the latest capability available. Another interesting option to rebuild the new solution in Oracle Planning & Budgeting Cloud Service (PBCS). PBCS is a subscription based solution that is hosted on Oracle’s Cloud platform and could be an excellent solution for the right client who is looking to rebuild their solution.

• Partial Rebuild in New Version and Upgrade – This involves only rebuilding certain components that are easier to rebuild and to upgrade the remaining components using one of the above mentioned processes. I usually recommend this when going from Financial Data Quality Management (FDM) classic to Financial Data Quality Management Enterprise Edition (FDMEE) or when transitioning to use different products.

Below I have highlighted the supported upgrade paths to 11.1.2.4 for reference to assist in identifying the strategy that best suits.

Path From Release …	To Release 11.1.2.4
11.1.2.x	Apply the maintenance release to move to Release 11.1.2.4. Note: For Financial Close Management, applying the maintenance release is supported only from Release 11.1.2.2 or 11.1.2.3. For Financial Management, applying the maintenance release is supported only from Release 11.1.2.1, 11.1.2.2 or 11.1.2.3. When you apply the maintenance release, you must install to the same machine as the previous installation. Out of place installations are not supported. To move to a new environment, first apply the maintenance release to move to Release 11.1.2.4, and then use Lifecycle Management to migrate the deployment to a new environment.
11.1.1.4.x	Upgrade to Release 11.1.2.3, and then apply the maintenance release to move to Release 11.1.2.4.
11.1.1.0.x to 11.1.1.3.x	Apply the maintenance release to move to Release 11.1.1.4, upgrade to Release 11.1.2.3, and then apply the maintenance release to move to Release 11.1.2.4.

Based on experience, my recommendation usually either falls in the staging approach, the in-place upgrade or rebuild territories. I recommend that people upgrade more regularly to avoid complex and costly upgrades. Maintenance release or in-place upgrades are quick to complete and are more like applying complex patching rather than upgrading. I also tend to really like the staging approach because it gives the simplicity of upgrading a single server instead of many in a distributed setup. This is extremely handy, especially when it comes to taking snapshot/clone backups along the way in the upgrade process. In the end whatever approach you choose is best determined by where you are and where you are trying to go and there will always be something that best suits your specific needs.

Final Cut-over

Unless you are using the in-place upgrade approach. You will end up with two systems, the old system and the upgraded system and you will need to transition to the new system at the end of the upgrade project. As can be seen above, a number of steps were required to get the system upgraded to the newest platform. Hyperion does not support migrations from one version to another using any standardized migration approach (for example life cycle manager). To migrate from one version to another one must follow the above mentioned migration steps. To cut-over to the new system here are the options available:

• Action the entire upgrade with the old system in read-only and go live on the new version
• Repeat the upgrade process once tested with old system in read-only and go-live
• Track metadata changes and sync metadata manually on new system. Data can be exported and imported from old to new system. In this approach some metadata that was updated during the upgrade project cannot be synced (supporting detail, cell text and commentary in planning).

Essentials vs Nice-To-Have

There is always the question of what to include as part of the project and a lot of time this is determined by the available budget. The analogy I like to draw is that of buying a car. When buying a car there are some standard things you get and then a number of things that you can get as optional extras (Airbags will be standard and leather may be optional).

There may also be an internal standards within organisations that might dictate what is a must vs what nice to have is. For example, for some clients IT policy may dictate that SSL must be implemented whilst others may not. It is thus essential to capture the requirement clearly in a technical solution design and ensure is in-line with budget expectations and implement accordingly.

For the sake of this blog post I am going to list what I like to ensure are essential and what I like to include as optional items.

Essential

Technical Design & Project Plan Hyperion Installation, Upgrade & Migration Tasks Integration Components Installation, Upgrade & Migration Tasks Apply Latest Patches Customized Start-up Scripts High-Level Base Tuning Hyperion Overnight Maintenance Solution Backup / Restore Solution Technical Testing (Includes Stress Testing) Functional Testing (Includes UAT and Go-Live Testing) Latest Client Tool Roll-out Training Documentation (Installation, Admin & Training Documentation)

Optional

SSL Setup High-Availability (Vertical or Horizontal Scaling/Clustering) Disaster Recovery Adopting New Available Functionality Improvements In-depth Tuning Single Sign-On

 

Setting Success Criteria

When upgrading a system, you need to define your success criteria. I have noted some of the basic ones below, but I must note that each upgrade may have different success criteria depending on the need:

• Successful Upgrade – The upgrade is successful and new version is rolled out to users, customer is happy.
• Solution & Upgrade Strategy Is Supported – Final solution is on a supported platform and the upgrade strategy to get there is fully supported. Final solution should also support for the latest client OS, Microsoft Office, Adobe and browser versions being rolled out / planned for the organization. The solution should also be in line with organization strategies for Server OS & Relational Server versions.
• Performance Improvements – Performance improvements are just the icing on the cake and probably the most noticeable given their direct impact on users and the user experience. The best way I can land this home with you is to draw the similarity of areas in life where we saw massive improvements and can’t imagine going backwards, for example, internet on dial-up speed vs now. Performance in Hyperion is usually out of the box software/hardware based or tuning the software/hardware to get the best performance out of them.
  • Software improvements – usually involve changes that have been made to the actual software that is out of the box or requires changes to be instated. For example, Hyperion 11.1.2.4 contains out of the box improvements such as Web Tier Performance, HFM Consolidation speeds and stable Essbase Hybrid (ASO/BSO) capability.
  • Hardware performance improvements – are usually benefits that are made by implementing a specific infrastructure configuration or implementing better hardware (better technology and specifications). Examples of areas that can make performance improvements:
    • Fast Disks or Solid State Disks
    • Implementing the Right Raid Configuration
    • Engineered Systems (Exalytics)
    • CPU (both number of cores and GHz are important)
• Meeting the Deadline / Budget – In short three key things:
  • Good Project Management – careful management of project tasks / timelines is essential. I always recommend planning carefully and avoiding a squeezed schedule where possible, as a risk is always what if something goes wrong in the upgrade process or what if we encounter an unexpected error. A good project plan will allow for such risks and a good project manager will manage them and the issues as they arise.
  • Good Technical Resources – This is like having a number 10 in a football pitch, he simply delivers. It’s as simple as this, when the game is tied at 0-0, he will score the winning goal or will find a way to get his team to glory. Likewise, a good technical resource has the knowledge and will not rest till he gets the job done. A good technical resource will be technically skilled, will have excellent communication skills and professionalism, can effectively translate the business requirement into a technical solution, will have strong business awareness and will be able to follow the project plan effectively and report deviations / issues as and when they happen to the project manager and to the client effectively.
  • Availability of IT & Business Resources – Business resources are required to test at various stages in the upgrade process. IT availability is required to prepare environments and provide ad-hoc support for Server and DBA tasks at various stages in the upgrade process.
• Minimizing Downtime – Think wrestling, specifically WWE, WWF or WCW, whatever you watched or didn’t watch. There are times when a team of wrestlers would tag team the opponent for a short duration of time. When it comes to minimizing downtime the Tag team strategy is my favorite. This can involve multiple resources working on the tasks at same time or in handover approach to maximize the work done in one day.
• Fixes for Existing Issues / New Functionality – For some older versions patches/fixes are no longer being released and in some cases even in more recent versions the fixes are applied only to the latest version. To give an example 11.1.2.1 is not seeing any recent patches, 11.1.2.3 still has patches being released. Having said that a lot of fixes / functionality may only made available in 11.1.2.4 version and its patches. The features and functionality is essential and at times the reason for upgrading, to compare the versions and see what is available, see the cumulative feature overview: https://apexapps.oracle.com/pls/apex/f?p=20620:1:0
• New Version Doesn’t Introduce Major Issues – Patch readmes should be reviewed to ensure any known issues are reviewed and no new major issues are introduced with the latest version. You should also check the defect finder available at: https://support.oracle.com/epmos/faces/DocumentDisplay?id=1292603.1

The Platform

Getting the final platform right is a key component of the upgrade. As mentioned prior, performance improvements and supportability are key, thus we need to spend the time to ensure the latest platform will deliver exactly that.

Changing the Platform (OS & Relational Types/Versions)

The platform you decide to host the final solution on will also have a great impact on the approach you take. Hyperion has the capability to migrate using the Life Cycle Manager (LCM) across Hyperion environments of the same version, but on heterogeneous platforms. You cannot however do upgrades to different platforms of type direct In-Place Upgrade (Maintenance Release) or Out-Of-Place Upgrade (Upgrade) as these are dependent the OS family and the relational family you are currently running. For example, if using Windows Server OS and Microsoft SQL Server Database you must use the same platform if using direct In-Place Upgrade (Maintenance Release) or Out-Of-Place Upgrade (Upgrade) approaches. There are three approaches you can consider if wanting to change platform (OS or Relational):

• Make the change on the current version – migrate to environment to desired platform or reconfigure system as required then perform the upgrade steps.
• Make the change on the target version – upgrade system on current platform then migrate to environment to desired platform or reconfigure system as required.
• Use the Staging upgrade approach (recommended approach) – refer to approach mentioned in the first section. Once the staging environment is upgraded it can be migrated to the new platform.

Physical vs Virtual Hosts

When it comes to this decision 90% of the time you may feel like to go physical is to try to move the wall of China and in conversations with IT, “will be the wall of China is not moving”. The reality is that IT at most organizations have spent the last 10 years moving all their platforms to centralized virtual solutions with the aim to reduce cost, increase flexibility, allow better scalability, improve/streamline backup and DR processes and in overall try to implement a better solution for the business. In more recent times with cloud/hosting solutions hitting the world by storm the concept is even further evolving by doing all of the above mentioned without having to maintain and support the hardware it sits on. The reality is this simple, the decision has two critical factors: support and performance.

• Support – When it comes to support Oracle only certify/fully support the Hyperion system to run on Oracle Virtual Machines. All other third-party virtual platforms get restricted/best effort support. What does this mean? In restricted support, Oracle may ask you to recreate the issue in a physical environment if they suspect the virtual layer is at fault. More information on this topic can be found at Oracle Support Document 588303.1 (Support for Oracle’s Hyperion Products in 3rd Party Virtualized Environments): https://support.oracle.com/epmos/faces/DocumentDisplay?id=588303.1. The reality is a lot of people are running Hyperion on platforms like VMWARE and are doing so very successfully without issue. I personally have never been asked to recreate the issue on an alternate physical platform and as stated have gotten best effort support. Please do not take this by any means that VMWARE is ok and the Oracle statement is irrelevant; this is more to say that if considering a virtual platform, this is a consideration that must be understood.
• Performance – In my experience physical is the ultimate way to guarantee performance of your system, especially with the appropriate architecture is applied. To understand this, you need a good understanding of physical and virtual platforms. Physical especially with dedicated local disks ensures all the resources on the server are dedicated to the software and OS components running on that host. Virtual is designed to share the resources of the host between many virtual machines (VMs) all living on the host or multiple hosts. This means that all the Software and OS components will be sharing server resources (RAM, CPU, DISK, NETWORK) with other software and OS components on other VMs.                                                                                                    The reality is that there is no clear cut answer that physical will give you better performance as the performance of virtual is dependent the underlying host resources and how much resources are dedicated to each virtual machine. Most virtualisation platforms will allow over allocation of CPU, meaning that you can have 6 VMs with 4 cores each on a host machine with 8 cores. In this example if more than 2 machines were busy, the other 4 VMs may need to wait for the host CPU to become available for them to process their required tasks. This is an example of over-allocation of resources and should be avoided to achieve best possible performance in a virtual environment. In the next example, we will also show how a virtual platform can offer better performance than a physical platform. If you had 3 physical machine with 8 Cores at 2.6GHz and 3 machines visualized on an Exalytics host with 60 cores at 2.6GHz, you can easily see how in this scenario the virtualised will easily outperform the physical machine.      However, the dark reality is that in most cases the virtualised solutions are in many cases are over-allocated, under-speced and are a one size fits all setup. Usually IT departments will ensure databases and ERP systems are given the best possible resources and everything else gets the one size fits all strategy. What many fail to realize is that Hyperion runs Essbase Server Database (a multidimensional database, scrap that, the world’s best multidimensional database). Essbase is extremely IO, CPU and memory intensive like its relational cousins. Therefore the key thing is to ensure that we treat Essbase as an enterprise database server that shall be responsible for planning, budgeting and reporting across the business. With the exception of Hyperion Financial Management that stores and consolidates data in a relational database; all other Hyperion data is housed in Essbase. By ensuring that Essbase has the best we can give it will help ensure the ideal planning, budgeting and reporting solution outcome.                                                                                                                                                     In summary, when comparing apples with apples, three servers given the exact same specs on a virtual platform vs a physical platform, physical will achieve best possible performance as the resources will be dedicated to each of those machines and resource contention is not a concern. If the aim is to maximize performance and cost is not an option, consider Exalytics, Exalytics is an engineered system for Hyperion / BI with performance as the main focus. Exalytics is an easier sell to IT when trying to go physical as is a specialized hardware for the software being implemented. In saying all this provided a virtual environment is setup correctly, there is no reason why virtual platforms cannot provide good performance and be successful for enterprise Hyperion deployments. The key to all this is discussions and establishing good relations with IT and together working towards the ideal solution for the organization.

Sizing

When it comes to sizing an environment this is dependent on model complexity, expected application growth, number of users, expect user concurrency and the platform this is to be installed on.

It is incredibly hard to give a quick guide on how to size Hyperion environments given there are so many variables as defined above. So rather than exact details or sample deployment scenarios I will give guidelines I believe are good starting points.

Oracle’s Minimum / Default Starting Point:

The Oracle documentation provides the below as the minimum standard deployment starting point. This is not a bad starting point to start your blueprint for the architecture design. In my opinion, the below is valid only as a minimum starting point with exception of RAM on the Foundation Server. Out of the box each EPM Web Application product will use up to 4GB of memory. The 6GB recommendation on most servers will cater for the product and leave some room for the OS. The Foundation Server is made up of a number of products such as Foundation Web Application, Framework Services, Framework Web Application, Essbase Administration Services Web, Provider Services Web, Financial Reporting Web, Web Analysis Web, Enterprise Performance Management Architect, Oracle HTTP Server and Weblogic Application Server. From my experience running all these component on a single server would require more than 10GB. To fit products in 10GB you would need to reduce the maximum memory for all web applications via adjusting the JVM –Xmx parameter and I do not recommend this.

EPM System Instance/Server	Processor	Memory
Foundation	4 core	10 GB
Planning	4 core	6 GB
Essbase	4 core	6 GB
Financial Management	4 core	6 GB
Profitability	4 core	6 GB
Strategic Finance	4 core	6 GB
FDMEE	4 core	6 GB

Platform Choice:

• <100 User / Average Complexity – Consider Virtual / Physical / Hybrid (Essbase Physical)
• 100+ Users / Complex Models – Consider a Dedicated Virtual / Hybrid (Essbase Physical) / Physical / Exalytics
• 300+ User / Complex Models – Consider Physical / Exalytics

Number of Servers:

• Virtual – More Servers, less consolidation due to resource contention.
• Physical –
• Exalytics – Excellent for consolidations, in most cases all products can live on one Server.

Server Resources:

• Virtual – Particularly in non-dedicated resource allocation, avoid high resource allocation of server resources to avoid resource contention. This is more relevant to CPU more than other server resources.
  • CPU – Do not allocate more than 4 cores as can cause resource contention
  • CPU – Avoid over allocated hosts.
  • DISK – Dedicated LUNs / Fast disk where possible

• Physical –
  • CPU (both number of cores and GHz are important)
  • PRODUCT SPLIT – Keep Essbase on its on host
  • DISK – Local Disks / Fast Disk / As Fast as possible for Essbase Data.
  • Exalytics – ensure the flash kit is included to maximize Essbase performance.

In summary

I hope this has shed some in-depth light on Hyperion upgrades and has helped arm you with knowledge for the journey ahead. I wish you best of success, this is Ahmed Hafez, signing out.

May the force be with you.

Advertisement