Working with OMs

dsTest publishes Operational Measurements (OMs) at specified intervals while any object that generates measurements is present, and it writes those values to an SQLite database file located in the directory from which dsTest was launched. See Working with OM Databases for more information on the proper handling of database files and how you can control the number and size of those files. We offer several reporting options - OM graphs, tabular reports, SmartReport, and HTML test results with dsClient Desktop along with a utility on your dsTest platform that converts the database values to CSV files. You can also query OM values in real time with dsClient Terminal or with dsTest's RESTful API for automated testing. This topic focuses on how dsTest collects measurements, the different types of measurements and how to calculate meaningful values, and how to craft OM query commands for an automated test.

dsTest publishes OMs at one-second intervals by default and you can adjust this timing with the report interval command. Measurement collecting and reporting runs at a lower priority than test transaction processing, which can result in skipped intervals when running at or near the capacity of the test platform. Lengthening the report interval to 10-15 seconds in this scenario will help to ensure that measurement values are posted to the interval in which the events they record occurred.

If your test creates 1,000 or more nodes of the same type, dsTest will automatically summarize the measurement values from all node instances into one structure for that node type.

OMs specific to your test configuration are generated by subscriber databases and by the application and transport/socket objects for each node in your test. You can view the measurements generated for applications and interfaces in the command section of the schema documentation (e.g., S6 OMs, Diameter OMs, and socket OMs). The types of measurements generated by subscriber databases, however, tend to be dynamically created based on your test configuration: a SmartProfile generates custom measurements based on your SmartFlow or SmartEvents message names, state names, and event handlers while a PCC Profile generates custom measurements based on the names of your CDR templates. dsTest also generates server-level diagnostic measurements to aid in troubleshooting.

Item	Description
ID	An integer key that is unique for each measurement
Name	The measurement's name
Type	The measurement's classification
Description	A short description of the measurement's value

As noted in the table above, measurements are classified by type. A measurement's type determines how its value should be interpreted as well as giving an indication of what type of data is stored with that measurement. In some cases further calculations are required to arrive at the final value. Our reporting options perform those calculations - you will only need to use the information below if you will be querying OMs during an automated test.

EVENT ACCUMULATOR: accumulates specific event occurrences over the lifetime of an object.

ERROR ACCUMULATOR: accumulates specific error occurrences over the lifetime of an object.

DURATION ACCUMULATOR: accumulates the elapsed time, in microseconds, to process all transactions of a particular type in a single interval. To calculate the final value, divide this value by the number of those transactions processed during the interval. This value includes network latency when recorded from the node that initiates a transaction.

DURATION MINIMUM: records the minimum value encountered while collecting the values of a DURATION ACCUMULATOR measurement for an interval.

DURATION MAXIMUM: records the maximum value encountered while collecting the values of a DURATION ACCUMULATOR measurement for an interval.

VARIANCE ACCUMULATOR: accumulates the cumulative square of the difference of each duration that expires in this interval from the mean. To calculate standard deviation, take the square root of the result of this value divided by the number of transactions in the interval (sqrt(variance/transactions)).

VALUE LOG: these measurements report per-interval values and are reset to zero when an interval expires. This type of measurement is often used for diagnostic OMs from the server level to the application and socket layers. Socket queue depths, transactions available, and requests pending are all examples of value log measurements.

Measurement categories group OMs by operation or function, allowing data to be summarized across applications and protocols regardless of which applications are involved in your test. You could, for example, calculate aggregate TPS or throughput. SmartReport makes use of OM categories to do just that. This section will be useful to you if you plan to build these types of summaries into your automated testing. If this is the case, you can use the category command to dynamically learn the categories that are associated with specific OMs, or you can query the omIndex table from an existing OM database and build the results into your script (category assignment is static and is only changed if an OM is mis-categorized). You can use the diag om command, as explained below, to summarize measurement values by category.

The matrix below shows the relationships of all measurement categories. To form the actual category stored in the database, combine the value in the column header with the value in the row header and then add one of the category details from the intersecting cell using underscores as delimiters (e.g., APPLICATION_STATUS_ATTEMPTS would summarize transaction attempts across all applications). Note that PROTOCOL refers to the application's protocol and covers, for example, Diameter Base transactions and protocol dictionary validation failures while TRANSPORT refers to the transport layer and socket - TCP or SCTP in the case of Diameter, for example.

	APPLICATION	PROTOCOL	TRANSPORT	USER_DATA	SERVER
ACTION	TARGET_RATE, ATTEMPTS, FAILURES, DELAYS, NO_SUBSCRIBER
STATUS	ATTEMPTS, SUCCESSES, FAILURES, RESETS	ATTEMPTS, SUCCESSES, FAILURES, RESETS	ATTEMPTS, SUCCESSES, FAILURES		RESOURCES_AVAILABLE, RESOURCES_USED
MESSAGES	NOTIFICATIONS_RCVD, NOTIFICATIONS_SENT, REQUESTS_RCVD, REQUESTS_SENT, RESPONSES_RCVD, RESPONSES_SENT, TRANSPORT_RCVD, TRANSPORT_SENT	NOTIFICATIONS_RCVD, NOTIFICATIONS_SENT, REQUESTS_RCVD, REQUESTS_SENT, RESPONSES_RCVD, RESPONSES_SENT, TRANSPORT_RCVD, TRANSPORT_SENT	RCVD, SENT	RCVD, SENT
LATENCY	DELAY, DURATION, MAX, MIN, VARIANCE	DURATION, MAX, MIN, VARIANCE	DELAY, MAX, MIN, VARIANCE	DELAY, MAX, MIN, VARIANCE
BYTES			RCVD, SENT	RCVD, SENT
RATES					THROTTLE_RATE_TRANSACTIONS
DIAGNOSTICS	ACTIVE, ERRORS, IDLE, NO_TRANSACTIONS, OVERRUN, QUEUE, RETRY, STATES	CONGESTION, DICTIONARY, ERRORS, OVERRUN, QUEUE, RETRY	ERRORS, OVERRUN, QUEUE, RETRY	ERRORS	THROTTLE_DELAYED, THROTTLE_DROPPED, TIMERS_EXPIRED

Not all OMs are assigned a category. In order to prevent double-counting, the measurements that record messages sent or received by message type are not assigned a category because the aggregate OMs messages sent and messages received are assigned the SENT and RCVD categories respectively.

You can query OM values, outside of a report, while your test is running through dsClient Terminal and through the RESTful API. There are two different ways to go about this: you can structure a command that begins with the parent object - node or subscriber database - and traverses the object hierarchy down to the object that owns the OM(s) of interest or you can use the diag om command to query any OM that is present in the current test. Each method has its advantages in particular situations.

When testing with 1,000 or more node emulators of the same type, the OMs associated with that node may be summarized into one node object. In that case, the OMs would be queried by the node element tag name without including a node name.

This method is best used when you're interested in a snapshot of OM values regarding a single object, and it lends itself to a command in dsClient Terminal's CLI or a URI for a REST GET request. If you're using the RESTful API, however, you can create a more complex hierarchical query by using a PUT request that includes multiple sub-elements such as last with non_zero_only or errors.

<node element tag name>[:<node name>] <object tag name>[:{index | <object name>}] om [last::1] [{errors::1 | count:{id | name} | non_zero_only::1}]

Identifies the applicable node by type and name. If :<node name> is not included, it is assumed that the name of the node is the same as the element tag name.

Identifies the application or interface from which measurements are collected, and the index or name of the object if multiple objects exist.

Displays a specific measurement identified by ID or Name. Use of the * wild card is supported when using name.

Display all of the OMs for the Diameter interface named s6 on the HSS node named core_hss.

CLI: spr:spr_1 subscriber_database subscriber_profiles SmartProfile SmartEvents om last::1

URI: /spr:spr_1/subscriber_database/subscriber_profiles/SmartProfile/SmartEvents/om/last::1

The diag om command, without any sub-elements, simply displays the server-level diagnostic measurements. It can also be used, however, to query any measurement and it offers options to summarize measurement values as well as the options described above that are available in the hierarchy query command. This command offers other options to manage OM generation and OM databases that are not discussed here but are described in the schema definition.

diag om objects returns a list of all OM-related objects currently loaded along with their IDs, their parents' IDs, and the number of OMs generated by each object.

diag om [last::1] [{{errors::1 | count:<OM specification> | non_zero_only::1} [all]} | categories | category:{id | name}}]

Displays only non-zero error diagnostic OMs. Displays all non-zero error OMs when all is included.

Displays a specific measurement defined by OM specification. This element's attributes listed below can be configured to summarize the values of an OM. You can include multiple count elements in one command (requires a REST PUT request).

object: the ID of the object that generates the OM returns only that OM's value; -1 returns the sum of all OMs with the specified OM ID; or

objects: a CSV list of object IDs returns the sum of only the OMs associated with a listed object

Displays only diagnostic OMs with values greater than zero. Displays all non-zero OMs when all is included.

Displays the summarized value of the specified category. You can include multiple category elements if you use a REST PUT request.