Test Methodology

Test Methodology

The tests were run against XenMobile Enterprise to establish benchmarks. In an effort to target both small and large scale deployments, 1,000 to 10,0000 devices were used in the measurements.

Workloads were created to simulate real-world use cases. These workloads were run for each test to study the effect on enrollment and logon rates. The objective of the tests was to obtain an optimal logon rate that was within the acceptable margin of error as outlined in . Logon rates are a critical factor in determining the hardware configuration recommendations for the infrastructure components.

The On-boarding (FTU) workload logon requests included auto discovery, authentication, and device registration operations. Application subscription, installation, and launch operations were uniformly distributed throughout the test period. This provided the best real-world simulation of user actions. At the conclusion of the test, the session was logged out. The Existing User workload logon requests included only authentication requests.

Workloads

On-Boarding (FTU) Workload

The On-boarding (FTU) workload is defined as the first time a user accesses the XenMobile environment. Operations included in this workload were:

Auto discovery

Enrollment

Authentication

Device registration

Application delivery (web, SaaS, and mobile MDX apps)

App subscription (including images and icon downloads)

Installation of the subscribed MDX apps

App launch (web, SaaS, and mobile MDX apps)

Minimal WorxMail and WorxWeb connections (VPN tunnels) — two connections

Installation of required apps through XenMobile

The workload parameters included:

1 device registration per device

1 enumeration per device

14 apps enumerated per device

4 Worx Store launches per device

4 web/SaaS app SSOs per device

1 MDX app downloaded per device

2 required app downloads

Existing Users Workload

The following table shows the Existing Users workload. This workload simulated a user using WorxMail and WorxWeb apps. This simulation was used to measure the NetScaler Gateway port's scalability within the XenMobile configuration. For the WorxWeb app, users were accessing internal web sites, which do not trigger XenMobile SSO. Operations in this mode included:

Authentication (NetScaler Gateway and XenMobile)

WorxMail and WorxWeb connections (VPN tunnels) — four connections

WorxApps Connection Profiles

1. Per session: 8 hours.

2. Type 1: Asymmetric send and receive with long lived connections (that is, WorxMail with a dedicated Microsoft Exchange mailbox connection).

3. Type 2: Asymmetric send and receive with connections that close and reopen after delays (that is, WorxWeb connections).

Note: Modifications to the connection details affect analysis results. For example, if the number of connections per user is increased, then the number of NetScaler Gateway sessions supported may be reduced.

WorxMail and WorxWeb Profiles

The following tables show the WorxMail and WorxWeb profile details.

Table 5. WorxMail Profile for a Medium Workload

Messages sent per day 20

Messages received per day 80

Messages read per day 80

Messages deleted per day 20

Average message size (KB) 200

Table 6. WorxWeb Profile for Medium Workload

Number of web apps launched 10

Number of web pages opened manually 10

Average number of request–response pairs per web app 100

Average size of request (bytes) 300

Average size of response (bytes) 1000

Configuration and Parameters

The following configurations were used when running the scalability tests:

NetScaler Gateway and load balancing (LB) virtual servers coexisted on the same NetScaler Gateway

appliance.

A 2048-bit key was used on NetScaler Gateway for SSL transactions.

Scaling XenMobile 10

Scaling XenMobile 10

Understanding the scale of your XenMobile infrastructure plays a significant role in how you decide to deploy and configure XenMobile. This article offers answers to common questions on determining the requirements for small to large scale enterprise deployments.

Performance and Scalability Guidelines

The data in this article are intended as guidelines for determining performance and scalability of a XenMobile infrastructure. The two key factors for determining how to configure your server and database are scalability (maximum users/devices) and logon rate.

Scalability is defined as the maximum number of concurrent users executing a defined workload. For more information on the flows used to load the XenMobile infrastructure, see .

Logon Rate is defined as the on-boarding of new users and the authentication of existing users. On-boarding rate is the maximum number of devices that can be enrolled on the environment for the first

time. Called First Time Use or FTU in this article, this data point is important when orchestrating a rollout strategy.

Existing user rate is the maximum number of users who authenticate to the environment, who have

already enrolled and connected with their device. These tests included creating sessions for already

enrolled users and the execution of WorxMail and WorxWeb apps.

System Configuration and Test Results

This section describes hardware configuration used and the results of running the On-boarding (FTU) workload and the Existing User workload scalability tests.

The following table defines the hardware and configuration recommendations for XenMobile when scaling from 1,000 to 100,000 devices. These guidelines are based on the test results and their associated workloads. The recommendations account for the acceptable margin of error as defined in.

Analysis of the test results led to these conclusions:

Logon rate is an important factor in determining the scalability of a system. In addition to the initial logon, logon rates are dependent upon the authentication time-out values configured in your environment. For instance, if you set the authentication time-out value too low, users must perform more frequent logon requests. Therefore, you need to clearly understand how time-out settings affect your environment.

An external database (SQL Server) with 128 GB of RAM, 300 GB of disk space, and 24 virtual CPUs was used for the tests and is recommended for production environments.

To achieve maximum scalability, CPU and RAM resources were increased on XenMobile.

The 10-node cluster configuration was the largest configuration validated. Scaling beyond 10 nodes requires an additional XenMobile implementation.

The preceding table shows the recommended on-boarding and existing user logon rates based on the XenMobile configuration, NetScaler Gateway appliance, cluster settings, and database. Use the data in this table to construct an optimal enrollment schedule for new deployments and returning user/device rates for existing deployments. The Configuration section relates enrollment and logon performance data to the appropriate hardware recommendations.

Note: You will experience the following if you exceed the recommended rates or hardware recommendations when sizing your system.

Enrollment or logon latency (round-trip time)

Total average latency: > 1.5 seconds

Average latency for a NetScaler Gateway logon: > 440 ms

Average latency for a Worx Store request: > 3 seconds

Physical performance degradation, such as CPU and memory exhaustion, was observed on the

infrastructure components when scalability limits were reached.

Invalid responses on the NetScaler Gateway and XenMobile appliances.

Slow XenMobile console response time.

The error percentage in the preceding figure includes the overall error experienced considering requests corresponding to every operation and is not limited to logons. The error percentage is within the acceptable limit for each test run as defined in.

The following figure shows the reference architecture for a small scale deployment. It is a standalone architecture that supports up to 10,000 devices.

The following figure shows the reference architecture for an enterprise deployment. It is a clustered architecture with SSL offload for MDM over HTTP that supports 10,000 or more devices.

Architecture Overview

The XenMobile components you deploy are based on the device or app management requirements of your organization. The components of XenMobile are modular and build on each other. For example, you want to give users in your organization remote access to mobile apps and you need to track the device types with which users connect. In this scenario, you would deploy XenMobile with NetScaler Gateway. XenMobile is where you manage apps and devices, and NetScaler Gateway enables users to connect to your network. 

Deploying XenMobile components: You can deploy XenMobile to enable users to connect to resources in your internal network in the following ways:

Connections to the internal network. If your users are remote, they can connect by using a VPN or micro VPN connection through NetScaler Gateway to access apps and desktops in the internal network. 

Device enrollment. Users can enroll mobile devices in XenMobile so you can manage the devices in the XenMobile console that connect to network resources. 

Web, SaaS, and mobile apps. Users can access their web, SaaS, and mobile apps from XenMobile through Worx Home. 

Windows-based apps and virtual desktops. Users can connect with Citrix Receiver or a web browser to access Windows-based apps and virtual desktops from StoreFront or the Web Interface.

To achieve some or all of these capabilities, Citrix recommends deploying XenMobile components in the following order: 

NetScaler Gateway. You can configure settings in NetScaler Gateway to enable communication with XenMobile, StoreFront, or the Web Interface by using the Quick Configuration wizard. Before using the Quick Configuration wizard in NetScaler Gateway, you must install XenMobile, StoreFront, or the Web Interface so that you can set up communication with it. 

XenMobile. After you install XenMobile, you can configure policies and settings in the XenMobile console that allow users to enroll their mobile devices. You also can configure mobile, web, and SaaS apps. Mobile apps can include apps from the Apple App Store or Google Play. Users can also connect to mobile apps you wrap with the MDX Toolkit and upload to the console. 

See the following eDocs for more information on policies, settings, enrollment, and apps:

Device Policies

Configuring XenMobile Client Settings

Configuring XenMobile Server Settings

Enrolling Users and Devices in XenMobile

Adding Apps to XenMobile

MDX Toolkit. The MDX Toolkit can securely wrap an app that was created within your organization or a mobile app made outside the company, such as the Citrix Worx apps. After you wrap an app, you then use the XenMobile console to add the app to XenMobile and change the policy configuration as needed. You can also add app categories, apply workflows, and deploy apps to delivery groups. See Wrapping Apps with the MDX Toolkit 10.0. 

StoreFront (optional). You can provide access to Windows-based apps and virtual desktops from StoreFront through connections with Receiver.

ShareFile Enterprise (optional). If you deploy ShareFile, you can enable enterprise directory integration through XenMobile, which acts as a Security Assertion Markup Language (SAML) identity provider. For more information about configuring identity providers for ShareFile, see the ShareFile support site.

XenMobile supports an integrated solution that provides device management, as well as app management through the XenMobile console. This section describes the reference architecture for the XenMobile deployment. 

The following figures illustrate different reference architectures for the XenMobile deployment. In the figures, the numbers on the connectors represent ports that must be opened to allow connections between the components. For a complete list of ports, see . 

Mobile device management (MDM) mode – XenMobile MDM Edition provides mobile device management for iOS, Android, Amazon, and Windows Phone (see ). In the recommended model, the XenMobile server is positioned in the DMZ with an optional NetScaler in front, which provides additional protection for XenMobile.

Mobile app management (MAM) mode – Mobile app management (MAM) supports iOS and Android devices, but not Windows Phone devices (see ). In the recommended deployment model, the XenMobile server is positioned with NetScaler Gateway in front, which provides additional rotection for XenMobile.

MAM with NetScaler Gateway (recommended deployment)

MDM and MAM modes – Using the MDM and MAM modes together provides mobile app and data management as well as mobile device management for iOS, Android, and Windows Phone (see

XenMobile 10). In the recommended deployment model, the XenMobile server is positioned in the DMZ with NetScaler Gateway in front, which provides additional protection for XenMobile.

Cluster deployment – In a production environment, Citrix recommends deploying the XenMobile solution in a cluster configuration for both scalability, as well as server redundancy purposes. Also, leveraging the NetScaler SSL Offload capability can further reduce the load on the XenMobile server and increase throughput.