--- title: "Analysis Workflows And Calculated Metrics" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Analysis Workflows And Calculated Metrics} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ## A little housekeeping This vignette uses the Adobe Analytics University Student Access account to show how to work with calculated metrics using R and the API. If you do not have access to the account, you can easily get it by going to the free LinkedIn Learning course, "Adobe Analytics Essential Training", and registering for access. If you haven't yet taken the course, it is well worth your time. Eric Matisoff does a great job walking through all the different amazing parts of Adobe Analytics, especially Analysis Workspace. Specifically, check out the third section of the first chapter to see a quick overview of the calculated metrics builder user interface in action. ```{r, include = FALSE, eval=FALSE} knitr::opts_chunk$set( collapse = FALSE, # eval = FALSE, warning = FALSE, comment = "#>" ) ``` ## Setup I won't be going into the any detail around authenticate in this article so if you are needing more help with that, make sure to check out the "Getting Started" vignette. With all that being set, let's get things going! First, we need to load the libraries and authenticate. ``` ## Load the packages needed library(adobeanalyticsr) library(tidyr) ## needed for some of the data wrangling library(jsonlite) ## Will help in visualizing calculated metrics definitions aw_auth('oauth') ##Check to make sure you have been logged into the correct account profile get_me() company_id <- 'adobea8cf' rsid <- 'igeo1xxpnwcidadobepm' ## Create a segment to be used farther down in the demo #create the segment rule segment_rule <- seg_rule(dimension = 'mobiledevicename', verb = 'exists') #build the segment and capture the segment id seginfo <- seg_build(name = 'NON Desktop traffic', description = 'This is a segment that filters out everything that is not associated with an identified mobile device', rules = list(segment_rule), create_seg = T) segid <- jsonlite::fromJSON(seginfo)$id ``` | globalCompanyId | companyName | | -------- | ------- | | adobea8cf | Adobe Analytics University Student Access | ## Retrieve multiple calculated metrics The following example shows a calculated metrics request for a response localized in US English, limited to the first page, and with the size of ten responses per page. ``` cms <- aw_get_calculatedmetrics(includeType = 'all', rsids = rsid, company_id = company_id) knitr::kable(head(cms), format = 'pipe') ``` |rsid |id |name |description |owner |polarity | precision|type | |:--------------------|:------------------------------------|:------------------------------|:----------------------------|:---------|:--------|---------:|:-------| |igeo1xxpnwcidadobepm |cm300010142_612a3e49262f3f1415082cee |Page | |200456743 |positive | 0|decimal | |igeo1xxpnwcidadobepm |cm300010142_61336bad44b1bd4f51a9e2ae |Bounce Rate (Bounces / Visits) | |200456743 |positive | 1|percent | |igeo1xxpnwcidadobepm |cm300010142_61336f20c7e6035b10558588 |Pages Count | |200456743 |positive | 0|decimal | |igeo1xxpnwcidadobepm |cm300010142_61416ee549ff383abc37c6ab |cart convert 1 |cart checkouts per page view |200483715 |positive | 4|decimal | |igeo1xxpnwcidadobepm |cm300010142_614179ec49ff383abc37c6c3 |Conversions | |200484198 |positive | 0|decimal | |igeo1xxpnwcidadobepm |cm300010142_614a8d494ba44e49bb383be5 |cart convert 1 | |200484193 |positive | 4|decimal | ## Retrieving a single calculated metric To retrieve a single calculated metric, include its id in the request. ``` cm <- aw_get_calculatedmetrics(filterByIds = cms$id[[1]], #add the calculated metric company_id = company_id, rsids = rsid, expansion = 'definition' #use the expansion argument to see the definition ) knitr::kable(cm) ``` |rsid |id |name |description |owner |polarity | precision|type |definition | |:--------------------|:------------------------------------|:----|:-----------|:---------|:--------|---------:|:-------|:-------------------| |igeo1xxpnwcidadobepm |cm300010142_612a3e49262f3f1415082cee |Page | |200456743 |positive | 0|decimal |visualization-group | ## Calculated metric management functions Calculated Metrics are comprised of several different mathematical functions that work on available metrics for a given report suite. ### Get all functions Returns a full list of calculated metric functions that the user can access. See the [functions documentation](https://experienceleague.adobe.com/docs/analytics/components/calculated-metrics/calcmetrics-reference/cm-functions.html?lang=en) for more information on available functions. ``` all_functions <- get_cm_functions(company_id = company_id) knitr::kable(head(all_functions[1:6])) ``` |id |category |persistable |name |description |definition | |:----------------------|:--------|:-----------|:-------------------------------|:-------------------------------------------------------------------------------------------------|:-----------| |col-sum |basic |TRUE |Column Sum |Adds all of the numeric values for a metric within a column (across the elements of a dimension). |calc-metric | |_new-lift-functional |internal |FALSE |NA |NA |calc-metric | |_lift-cumul-avg-impl |internal |FALSE |NA |NA |calc-metric | |_waskr-N-control |internal |FALSE |NA |NA |calc-metric | |ls-intercept-quadratic |advanced |TRUE |Quadratic regression: Intercept |Quadratic regression: Y = ( a + b X ) ^ 2, Returns a. |calc-metric | |_chi2-test-stat |internal |FALSE |NA |NA |calc-metric | ### Create a calculated metric The process of creating a calculated metric has been designed in a modular way to enable the systematic approach to building and maintaining complex calculated metrics in the most efficient way. The goal is to enhance analysis and data science workflows. The essential elements of a calculated metric are: function, formula, build. The following code chunks will illustrate the basics of interacting with each element. #### Create a function object Calculated Metrics are comprised of several different mathematical functions that work on available metrics for a given report suite. See the [functions documentation](https://experienceleague.adobe.com/docs/analytics/components/calculated-metrics/calcmetrics-reference/cm-functions.html?lang=en) for more information on available functions. Currently the package only supports 2 types of functions: 1. Any function that take a single metric - Examples of these functions include `Absolute Value (Row)` and `Column Maximum`. Both of which apply a function on a single metric. For a full list make sure to see the [functions documentation](https://experienceleague.adobe.com/docs/analytics/components/calculated-metrics/calcmetrics-reference/cm-functions.html). 2. The `Approximate Count Distinct (dimension)` function - Returns the approximated distinct count of dimension items for the selected dimension. The function uses the HyperLogLog (HLL) method of approximating distinct counts. It is configured to guarantee the value is within 5% of the actual value 95% of the time. ``` ?cm_function cm_func <- cm_function( func = "col-sum", metric = "visits", seg_filter = NULL, rsid = rsid, company_id = company_id ) jsonlite::toJSON(cm_func, pretty = T, auto_unbox = T) ``` { "func": "col-sum", "description": "Column Sum", "col": { "func": "metric", "name": "metrics/visits", "description": "Visits" } } #### Create a formula object A calculated metric formula takes to metric objects and applies an operator on them. You can nest formula or function objects within formulas to create complex operational containers. Keep in mind, unlike segment containers, these containers function like a math expression and determine the order of operations. ``` ?cm_formula cm_form <- cm_formula( operator = 'divide', metrics = list(cm_func, "singlepagevisits"), seg_filters = NA, #add segment filters for each metric if needed rsid = rsid, company_id = company_id ) jsonlite::toJSON(cm_form, pretty = T, auto_unbox = T) ``` { "func": "divide", "col1": { "func": "col-sum", "description": "Column Sum", "col": { "func": "metric", "name": "metrics/visits", "description": "Visits" } }, "col2": { "func": "metric", "name": "metrics/singlepagevisits", "description": "Single Page Visits" } } #### Create a calculated metric object Using the `cm_build` function gives the ability to create and validate simple and complex calculated metrics. If you are familiar with the calculated metrics user interface you should be able to quickly recognize what each aspect of the function arguments refer to but fo those who are less familiar the [calculated metrics builder documentation](https://experienceleague.adobe.com/docs/analytics/components/calculated-metrics/calcmetric-workflow/cm-build-metrics.html?lang=en#areas-of-the-calculated-metrics-builder) may be of great use. ``` ?cm_build cm_obj <- cm_build( name = 'Test Calculated Metric', description = 'Test cm description', formula = cm_form, seg_filter = NULL, polarity = "positive", precision = 0, type = "decimal", create_cm = FALSE, #should this be created in the UI debug = FALSE, rsid = rsid, company_id = company_id ) cm_obj ``` {"rsid":"igeo1xxpnwcidadobepm","name":"Test Calculated Metric","description":"Test cm description","definition":{"formula":{"func":"divide","col1":{"func":"col-sum","description":"Column Sum","col":{"func":"metric","name":"metrics/visits","description":"Visits"}},"col2":{"func":"metric","name":"metrics/singlepagevisits","description":"Single Page Visits"}},"version":[1,0,0],"func":"calc-metric"},"polarity":"positive","precision":0,"type":"decimal"} ### Validate a calculated metric object Because report suites can have different configurations, dimensions, or metrics, a calculated metric that is valid in one report suite may not be valid in another. To determine which calculated metric to use in different report suites, and why it may or may not be available, you can use the cm_validate. This endpoint allows you to POST a definition along with a target report suite id. The validate endpoint responds with compatibility information on the calculated metric. ``` ?cm_val cm <- cm_val(cm_obj) cm ``` [1] "The calculated metric definition IS VALID" Once yo determine make we can create the new calculated metric in the UI by setting the argument `create_cm = TRUE`. ``` #create the calculated metric in the UI cm_obj <- cm_build( name = 'Test Calculated Metric', description = 'Test cm description', formula = cm_form, seg_filter = NULL, #include an overall segment filter polarity = "positive", precision = 0, type = "decimal", create_cm = TRUE, #should this be created in the UI debug = FALSE, rsid = rsid, company_id = company_id ) knitr::kable(cm_obj) ``` {"rsid":"igeo1xxpnwcidadobepm","id":"cm300010142_6537f60dde0e706af6156fb1","name":"Test Calculated Metric","description":"Test cm description","isDeleted":false,"migratedIds":[],"internal":false,"owner":{"id":200654087},"hidden":false,"componentType":"calculatedMetric","polarity":"positive","precision":0,"type":"decimal","definition":{"formula":{"func":"divide","col1":{"func":"col-sum","description":"Column Sum","col":{"func":"metric","name":"metrics/visits","description":"Visits"}},"col2":{"func":"metric","name":"metrics/singlepagevisits","description":"Single Page Visits"}},"func":"calc-metric","version":[1,0,0]},"compatibility":{"identityMetrics":[{"identity":"metrics/visits"},{"identity":"metrics/singlepagevisits"}],"functions":["col-sum","divide"],"validator_version":"1.0.0","supported_products":["oberon"],"supported_schema":["schema_oberon","schema_frag"]},"legacyId":"","categories":["Calculated Metrics"],"modified":"2023-10-24T16:51:25Z"} ### Adding a segment filter On every level of a calculated metric, function, formula, and object, you can add an additional level of control by inserting a segment filter. ``` # use the new segment id to create a segment cm_obj_seg_filter <- cm_build( name = 'Test Segment Filtered Calculated Metric', description = 'Test cm description that includes a segment id', formula = cm_form, seg_filter = segid, #adding the segment here create_cm = FALSE, rsid = rsid, company_id = company_id ) cm_val(cm_obj_seg_filter) ``` [1] "The calculated metric definition IS VALID" Now that we know it is a valid segment, let's create it and pull the data along with the previous segment to see the difference in the data. ``` # use the new segment id to create a segment cm_obj_seg_filter <- cm_build( name = 'Test Segment Filtered Calculated Metric', description = 'Test cm description that includes a segment id', formula = cm_form, seg_filter = segid, #adding the segment here create_cm = TRUE, rsid = rsid, company_id = company_id ) cm_seg_filter <- jsonlite::fromJSON(cm_obj_seg_filter)$id cm_no_seg_filter <- jsonlite::fromJSON(cm_obj)$id df_cm <- aw_freeform_table(company_id = company_id, rsid = rsid, metrics = c(cm_no_seg_filter, cm_seg_filter), dimensions = 'daterangeday', prettynames = T) knitr::kable(df_cm) ``` |Day | Test Calculated Metric| Test Segment Filtered Calculated Metric| |:----------|----------------------:|---------------------------------------:| |2023-09-26 | 3707.0726| Inf| |2023-09-25 | 460.1371| 465.8842| |2023-09-27 | 401.8156| 406.0459| |2023-10-20 | 383.0642| 348.4961| |2023-10-14 | 373.7211| 348.4961| ### Update a calculated metric To update the name or description it is as simple as adding them within a list function of the `updates` argument. ``` ?cm_update cm_updated <- cm_update( id = cm_no_seg_filter, updates = list(name = "new name", description = "this is a new description"), locale = "en_US", debug = FALSE, company_id = company_id ) # Name Change c(`old name` = jsonlite::fromJSON(cm_obj)$name,`new name` = cm_updated$name) # Description Change c(`old description` = jsonlite::fromJSON(cm_obj)$description,`new description` = cm_updated$description) ``` old name new name "Test Calculated Metric" "new name" old description new description "Test cm description" "this is a new description" For updating an existing calculated metrics within the definition there is a more complex solution that needs to be done. ``` cm_updated <- aw_get_calculatedmetrics(filterByIds = cm_updated$id, expansion = 'definition') #change the col1 values to reflect a new function `mean` cm_updated$definition$formula$col1$func <- 'mean' cm_updated$definition$formula$col1$description <- 'Mean' cm_updated$definition$formula$col1$`include-zeros` <- FALSE cm_update_complete <- cm_update( id = cm_updated$id, updates = jsonlite::unbox(cm_updated), locale = "en_US", debug = FALSE, company_id = company_id ) ``` `cm_update_complete$definition$formula$col1$func` [1] "mean" `cm_update_complete$definition$formula$col1$description` [1] "Mean" ### Copy a Calculated Metric ``` cm_var <- aw_get_calculatedmetrics(filterByIds = jsonlite::fromJSON(cm_obj)$id, expansion = 'definition') jsonlite::toJSON(cm_var) copy_res <- cm_copy(cm_id = cm_var$id, name = 'Here I copy the cm', description = 'I want to add a new description', polarity = 'negative', precision = 2, type = 'percent', create_cm = T) ``` `copy_res` "{"rsid":"igeo1xxpnwcidadobepm","id":"cm300010142_6537f722de0e706af6156fb3","name":"Here I copy the cm","description":"I want to add a new description","isDeleted":false,"migratedIds":[],"internal":false,"owner":{"id":200654087},"hidden":false,"componentType":"calculatedMetric","polarity":"negative","precision":2,"type":"percent","definition":{"formula":{"func":"divide","col1":{"func":"mean","description":"Mean","col":{"func":"metric","name":"metrics/visits","description":"Visits"},"include-zeros":false},"col2":{"func":"metric","name":"metrics/singlepagevisits","description":"Single Page Visits"}},"func":"calc-metric","version":[1,0,0]},"compatibility":{"identityMetrics":[{"identity":"metrics/visits"},{"identity":"metrics/singlepagevisits"}],"functions":["mean","divide"],"validator_version":"1.0.0","supported_products":["oberon"],"supported_schema":["schema_oberon","schema_frag"]},"legacyId":"","categories":["Calculated Metrics"],"modified":"2023-10-24T16:56:02Z"}" `cm_val(copy_res)` [1] "The calculated metric definition IS VALID" ### Delete calculated metrics ``` ?cm_delete deleted <- cm_delete( cm_id = cm_no_seg_filter, warn = FALSE, locale = "en_US", debug = FALSE, rsid = rsid, company_id = company_id ) ``` `r message('success: cm300010142_6537f60dde0e706af6156fb1 has been deleted')` ## Conclusion The set of calculated metric functions are setup to enable the end user to easily incorporate calculated metrics into the analysis and data science workflows without needing to interact with the UI. At the same time, the application of integrating the API in workflows enables a broader set of analytics users to incorporate the results and ongoing analysis in the powerful and versatile Analysis Workspace. Make sure to add an issue in Github or add a pull request if you find additional opportunities of development that would enhance the ability to analyze and communicate the results using calculated metrics in your workflows.