---
title: "Creating ADVFQ"
output:
  rmarkdown::html_vignette:
vignette: >
  %\VignetteIndexEntry{Creating ADVFQ}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---
  
```{r setup, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

# Introduction

This article describes creating an ADVFQ ADaM with Visual Functioning Questionnaire
data for ophthalmology endpoints. It is to be used in conjunction with the article on [creating a BDS dataset from SDTM](https://pharmaverse.github.io/admiral/articles/bds_finding.html). As such, derivations and processes that are not specific to ADVFQ are mostly absent, and the user is invited to consult the aforementioned article for guidance. 

The full, open-source VFQ questionnaire can be accessed [here](https://www.nei.nih.gov/about/education-and-outreach/outreach-materials/visual-function-questionnaire-25).

**Note**: *All examples assume CDISC SDTM and/or ADaM format as input unless
otherwise specified. Also, some of the example datasets in this vignette contain more records than are displayed by default, but the number of displayed records can be expanded using the selectors at the bottom.* 

## Dataset Contents

`{admiralophtha}` suggests to populate ADVFQ solely with VFQ-related records. Any other questionnaire data should be placed in separate datasets (e.g. ADQS).

The records in ADVFQ can be categorized in four groups:

1. The __original__/__raw__ records coming from the VFQ itself. These include both the Base items in the standard VFQ-25 and the Optional items which can be added to form the VFQ-39. Importantly, not all of the items are measured on the same scale.
1. The __transformed__ records, which are in a one-to-one correspondence with the original records and serve to recode the latter on the same 0 - 100 scale.
1. The __composite scores by category__, which are means of all the transformed records from within a category, e.g. "Near activities score". These composite scores can be calculated including or excluding the Optional items.
1. The __overall composite scores__, which are means of the composite scores, again including or excluding the Optional items.

## Required Packages

The examples of this vignette require the following packages.

```{r, warning=FALSE, message=FALSE}
library(dplyr)
library(admiral)
library(pharmaversesdtm)
library(admiraldev)
library(admiralophtha)
library(stringr)
```

# Programming Workflow

* [Reading In Data and Setting Up Lookup Tables](#setup_lookup)
* [Initial Set Up and Mapping the Raw Records](#mapping_raw)
* [Deriving the Transformed Parameters](#deriving_transformed) 
* [Deriving the Composite Parameters](#deriving_composite)
* [Further Derivations of Standard BDS Variables](#further)
* [Example Script](#example) 

## Reading In Data and Setting Up Lookup Tables {#setup_lookup}

To start, all datasets needed for the creation of the VFQ analysis dataset
should be read into the environment. For the purpose of demonstration we shall use the `{pharmaversesdtm}` `qs_ophtha` and the `{admiral}` ADSL test datasets. Note that the former only contains VFQ records.

```{r} 
data("admiral_adsl")
data("qs_ophtha")

adsl <- admiral_adsl
qs <- qs_ophtha
```

Next, it will prove useful to set up lookup tables ahead of time for the `PARAM`/`PARAMCD` and `PARCATy` variables for the original, transformed and composite parameters. This is so that when the latter two are derived later in the script, we can simply assign `PARAMCD` (or one of the `PARCATy`s) and then perform a merge with the lookup tables to associate to them the remaining variables as well. __Please note that the mappings described through these lookup tables may vary from company to company - this is just what `{admiralophtha}` suggests.__

For convenience, we suggest setting up three separate lookup tables. You can peruse how the tables are structured below - for the full set up code, please visit the [ad_advfq.R](https://github.com/pharmaverse/admiralophtha/blob/main/inst/templates/ad_advfq.R) template. There is a special importance for the contents of `PARCAT4` since this determines the categories which are then averaged within the [composite parameters](#deriving_composite). You can differentiate the Base Items from the Optional ones using `PARCAT5`, and the original, transformed and composite parameters using `PARCAT2`.

```{r, eval=TRUE, echo=FALSE} 
## Original parameters (note PARAM, PARAMCD are identical to QSTEST, QSTESTCD) ----
param_lookup_original <- tribble(
  ~QSTESTCD,  ~PARAM,                                ~PARCAT3,                ~PARCAT4,                              ~PARCAT5,
  "VFQ101",   "Your Overall Health Is",              "general health",        "General Health",                      "Base Item",
  "VFQ102",   "Eyesight Using Both Eyes Is",         "general vision",        "General Vision",                      "Base Item",
  "VFQ103",   "How Often You Worry About Eyesight",  "well-being/distress",   "Vision Specific: Mental Health",      "Base Item",
  "VFQ104",   "How Much Pain in and Around Eyes",    "ocular pain",           "Ocular Pain",                         "Base Item",
  "VFQ105",   "Difficulty Reading Newspapers",       "near vision",           "Near Activities",                     "Base Item",
  "VFQ106",   "Difficulty Doing Work/Hobbies",       "near vision",           "Near Activities",                     "Base Item",
  "VFQ107",   "Difficulty Finding on Crowded Shelf", "near vision",           "Near Activities",                     "Base Item",
  "VFQ108",   "Difficulty Reading Street Signs",     "distance vision",       "Distance Activities",                 "Base Item",
  "VFQ109",   "Difficulty Going Down Step at Night", "distance vision",       "Distance Activities",                 "Base Item",
  "VFQ110",   "Difficulty Noticing Objects to Side", "peripheral vision",     "Peripheral Vision",                   "Base Item",
  "VFQ111",   "Difficulty Seeing How People React",  "social fx",             "Vision Specific: Social Functioning", "Base Item",
  "VFQ112",   "Difficulty Picking Out Own Clothes",  "color vision",          "Color Vision",                        "Base Item",
  "VFQ113",   "Difficulty Visiting With People",     "social fx",             "Vision Specific: Social Functioning", "Base Item",
  "VFQ114",   "Difficulty Going Out to See Movies",  "distance vision",       "Distance Activities",                 "Base Item",
  "VFQ115",   "Are You Currently Driving",           "driving (filter item)", NA_character_,                         "Base Item",
  "VFQ115A",  "Never Driven or Given Up Driving",    "driving (filter item)", NA_character_,                         "Base Item",
  "VFQ115B",  "Main Reason You Gave Up Driving",     "driving (filter item)", NA_character_,                         "Base Item",
  "VFQ115C",  "Difficulty Driving During Daytime",   "driving",               "Driving",                             "Base Item",
  "VFQ116",   "Difficulty Driving at Night",         "driving",               "Driving",                             "Base Item",
  "VFQ116A",  "Driving in Difficult Conditions",     "driving",               "Driving",                             "Base Item",
  "VFQ117",   "Accomplish Less Than You Would Like", "role limitations",      "Vision Specific: Role Difficulties",  "Base Item",
  "VFQ118",   "Limited in How Long You Can Work",    "role limitations",      "Vision Specific: Role Difficulties",  "Base Item",
  "VFQ119",   "Eye Pain Keep From Doing What Like",  "ocular pain",           "Ocular Pain",                         "Base Item",
  "VFQ120",   "I Stay Home Most of the Time",        "dependency",            "Vision Specific: Dependency",         "Base Item",
  "VFQ121",   "I Feel Frustrated a Lot of the Time", "well-being/distress",   "Vision Specific: Mental Health",      "Base Item",
  "VFQ122",   "Much Less Control Over What I Do",    "well-being/distress",   "Vision Specific: Mental Health",      "Base Item",
  "VFQ123",   "Rely Too Much on What Others Tell",   "dependency",            "Vision Specific: Dependency",         "Base Item",
  "VFQ124",   "I Need a Lot of Help From Others",    "dependency",            "Vision Specific: Dependency",         "Base Item",
  "VFQ125",   "Worry I'll Do Embarrassing Things",   "well-being/distress",   "Vision Specific: Mental Health",      "Base Item",
  "VFQ1A01",  "Rate Your Overall Health",            "general health",        "General Health",                      "Optional Item",
  "VFQ1A02",  "Rate Your Eyesight Now",              "general vision",        "General Vision",                      "Optional Item",
  "VFQ1A03",  "Difficulty Reading Small Print",      "near vision",           "Near Activities",                     "Optional Item",
  "VFQ1A04",  "Difficulty Figure Out Bill Accuracy", "near vision",           "Near Activities",                     "Optional Item",
  "VFQ1A05",  "Difficulty Shaving or Styling Hair",  "near vision",           "Near Activities",                     "Optional Item",
  "VFQ1A06",  "Difficulty Recognizing People",       "distance vision",       "Distance Activities",                 "Optional Item",
  "VFQ1A07",  "Difficulty Taking Part in Sports",    "distance vision",       "Distance Activities",                 "Optional Item",
  "VFQ1A08",  "Difficulty Seeing Programs on TV",    "distance vision",       "Distance Activities",                 "Optional Item",
  "VFQ1A09",  "Difficulty Entertaining Friends",     "social fx",             "Vision Specific: Social Functioning", "Optional Item",
  "VFQ1A11A", "Do You Have More Help From Others",   "role limitations",      "Vision Specific: Role Difficulties",  "Optional Item",
  "VFQ1A11B", "Limited in Kinds of Things Can Do",   "role limitations",      "Vision Specific: Role Difficulties",  "Optional Item",
  "VFQ1A12",  "Often Irritable Because Eyesight",    "well-being/distress",   "Vision Specific: Mental Health",      "Optional Item",
  "VFQ1A13",  "I Don't Go Out of My Home Alone",     "dependency",            "Vision Specific: Dependency",         "Optional Item"
) %>%
  mutate(
    PARAMCD = QSTESTCD,
    PARCAT1 = "VFQ-25 INTERVIEWER ADMINISTERED",
    PARCAT2 = "Original Items"
  ) %>%
  select(QSTESTCD, PARAM, PARAMCD, PARCAT1, PARCAT2, PARCAT3, PARCAT4, PARCAT5)

## Transformed parameters ----
param_lookup_transformed <- tribble(
  ~PARAMCD,  ~PARAM,                                              ~PARCAT3,              ~PARCAT4,                              ~PARCAT5,
  "QR01",    "Transformed - Your Overall Health Is",              "general health",      "General Health",                      "Base Item",
  "QR02",    "Transformed - Eyesight Using Both Eyes Is",         "general vision",      "General Vision",                      "Base Item",
  "QR03",    "Transformed - How Often You Worry About Eyesight",  "well-being/distress", "Vision Specific: Mental Health",      "Base Item",
  "QR04",    "Transformed - How Much Pain in and Around Eyes",    "ocular pain",         "Ocular Pain",                         "Base Item",
  "QR05",    "Transformed - Difficulty Reading Newspapers",       "near vision",         "Near Activities",                     "Base Item",
  "QR06",    "Transformed - Difficulty Doing Work/Hobbies",       "near vision",         "Near Activities",                     "Base Item",
  "QR07",    "Transformed - Difficulty Finding on Crowded Shelf", "near vision",         "Near Activities",                     "Base Item",
  "QR08",    "Transformed - Difficulty Reading Street Signs",     "distance vision",     "Distance Activities",                 "Base Item",
  "QR09",    "Transformed - Difficulty Going Down Step at Night", "distance vision",     "Distance Activities",                 "Base Item",
  "QR10",    "Transformed - Difficulty Noticing Objects to Side", "peripheral vision",   "Peripheral Vision",                   "Base Item",
  "QR11",    "Transformed - Difficulty Seeing How People React",  "social fx",           "Vision Specific: Social Functioning", "Base Item",
  "QR12",    "Transformed - Difficulty Picking Out Own Clothes",  "color vision",        "Color Vision",                        "Base Item",
  "QR13",    "Transformed - Difficulty Visiting With People",     "social fx",           "Vision Specific: Social Functioning", "Base Item",
  "QR14",    "Transformed - Difficulty Going Out to See Movies",  "distance vision",     "Distance Activities",                 "Base Item",
  "QR15C",   "Transformed - Main Reason You Gave Up Driving",     "driving",             "Driving",                             "Base Item",
  "QR16",    "Transformed - Difficulty Driving at Night",         "driving",             "Driving",                             "Base Item",
  "QR16A",   "Transformed - Driving in Difficult Conditions",     "driving",             "Driving",                             "Base Item",
  "QR17",    "Transformed - Accomplish Less Than You Would Like", "role limitations",    "Vision Specific: Role Difficulties",  "Base Item",
  "QR18",    "Transformed - Limited in How Long You Can Work",    "role limitations",    "Vision Specific: Role Difficulties",  "Base Item",
  "QR19",    "Transformed - Eye Pain Keep From Doing What Like",  "ocular pain",         "Ocular Pain",                         "Base Item",
  "QR20",    "Transformed - I Stay Home Most of the Time",        "dependency",          "Vision Specific: Dependency",         "Base Item",
  "QR21",    "Transformed - I Feel Frustrated a Lot of the Time", "well-being/distress", "Vision Specific: Mental Health",      "Base Item",
  "QR22",    "Transformed - Much Less Control Over What I Do",    "well-being/distress", "Vision Specific: Mental Health",      "Base Item",
  "QR23",    "Transformed - Rely Too Much on What Others Tell",   "dependency",          "Vision Specific: Dependency",         "Base Item",
  "QR24",    "Transformed - I Need a Lot of Help From Others",    "dependency",          "Vision Specific: Dependency",         "Base Item",
  "QR25",    "Transformed - Worry I'll Do Embarrassing Things",   "well-being/distress", "Vision Specific: Mental Health",      "Base Item",
  "QRA01",   "Transformed - Rate Your Overall Health",            "general health",      "General Health",                      "Optional Item",
  "QRA02",   "Transformed - Rate Your Eyesight Now",              "general vision",      "General Vision",                      "Optional Item",
  "QRA03",   "Transformed - Difficulty Reading Small Print",      "near vision",         "Near Activities",                     "Optional Item",
  "QRA04",   "Transformed - Difficulty Figure Out Bill Accuracy", "near vision",         "Near Activities",                     "Optional Item",
  "QRA05",   "Transformed - Difficulty Shaving or Styling Hair",  "near vision",         "Near Activities",                     "Optional Item",
  "QRA06",   "Transformed - Difficulty Recognizing People",       "distance vision",     "Distance Activities",                 "Optional Item",
  "QRA07",   "Transformed - Difficulty Taking Part in Sports",    "distance vision",     "Distance Activities",                 "Optional Item",
  "QRA08",   "Transformed - Difficulty Seeing Programs on TV",    "distance vision",     "Distance Activities",                 "Optional Item",
  "QRA09",   "Transformed - Difficulty Entertaining Friends",     "social fx",           "Vision Specific: Social Functioning", "Optional Item",
  "QR1A11A", "Transformed - Do You Have More Help From Others",   "role limitations",    "Vision Specific: Role Difficulties",  "Optional Item",
  "QR1A11B", "Transformed - Limited in Kinds of Things Can Do",   "role limitations",    "Vision Specific: Role Difficulties",  "Optional Item",
  "QR1A12",  "Transformed - Often Irritable Because Eyesight",    "well-being/distress", "Vision Specific: Mental Health",      "Optional Item",
  "QR1A13",  "Transformed - I Don't Go Out of My Home Alone",     "dependency",          "Vision Specific: Dependency",         "Optional Item"
) %>%
  mutate(
    PARCAT1 = "VFQ-25 INTERVIEWER ADMINISTERED",
    PARCAT2 = "Transformed - Original Items"
  ) %>%
  select(PARAM, PARAMCD, PARCAT1, PARCAT2, PARCAT3, PARCAT4, PARCAT5)

## Composite parameters ----
param_lookup_composite <- tribble(
  ~PARAMCD,   ~PARAM,                                                             ~PARCAT3,      ~PARCAT4,                              ~PARCAT5,
  "QSBGH",    "General Health Score",                                             NA_character_, "General Health",                      "VFQ-25",
  "QSBGV",    "General Vision Score",                                             NA_character_, "General Vision",                      "VFQ-25",
  "QSBNA",    "Near Activities Score",                                            NA_character_, "Near Activities",                     "VFQ-25",
  "QSBDA",    "Distance Activities Score",                                        NA_character_, "Distance Activities",                 "VFQ-25",
  "QSBSF",    "Vision Specific: Social Functioning Score",                        NA_character_, "Vision Specific: Social Functioning", "VFQ-25",
  "QSBCV",    "Color Vision Score",                                               NA_character_, "Color Vision",                        "VFQ-25",
  "QSBPV",    "Peripheral Vision Score",                                          NA_character_, "Peripheral Vision",                   "VFQ-25",
  "QSBDR",    "Driving Score",                                                    NA_character_, "Driving",                             "VFQ-25",
  "QSBRD",    "Vision Specific: Role Difficulties Score",                         NA_character_, "Vision Specific: Role Difficulties",  "VFQ-25",
  "QSBOP",    "Ocular Pain Score",                                                NA_character_, "Ocular Pain",                         "VFQ-25",
  "QSBDP",    "Vision Specific: Dependency Score",                                NA_character_, "Vision Specific: Dependency",         "VFQ-25",
  "QSBMH",    "Vision Specific: Mental Health Score",                             NA_character_, "Vision Specific: Mental Health",      "VFQ-25",
  "QSOGH",    "General Health Score (incl. Optional Items)",                      NA_character_, "General Health",                      "VFQ-39",
  "QSOGV",    "General Vision Score (incl. Optional Items)",                      NA_character_, "General Vision",                      "VFQ-39",
  "QSONA",    "Near Activities Score (incl. Optional Items)",                     NA_character_, "Near Activities",                     "VFQ-39",
  "QSODA",    "Distance Activities Score (incl. Optional Items)",                 NA_character_, "Distance Activities",                 "VFQ-39",
  "QSOSF",    "Vision Specific: Social Functioning Score (incl. Optional Items)", NA_character_, "Vision Specific: Social Functioning", "VFQ-39",
  "QSOCV",    "Color Vision Score (incl. Optional Items)",                        NA_character_, "Color Vision",                        "VFQ-39",
  "QSOPV",    "Peripheral Vision Score (incl. Optional Items)",                   NA_character_, "Peripheral Vision",                   "VFQ-39",
  "QSODR",    "Driving Score (incl. Optional Items)",                             NA_character_, "Driving",                             "VFQ-39",
  "QSORD",    "Vision Specific: Role Difficulties Score (incl. Optional Items)",  NA_character_, "Vision Specific: Role Difficulties",  "VFQ-39",
  "QSOOP",    "Ocular Pain Score (incl. Optional Items)",                         NA_character_, "Ocular Pain",                         "VFQ-39",
  "QSODP",    "Vision Specific: Dependency Score (incl. Optional Items)",         NA_character_, "Vision Specific: Dependency",         "VFQ-39",
  "QSOMH",    "Vision Specific: Mental Health Score (incl. Optional Items)",      NA_character_, "Vision Specific: Mental Health",      "VFQ-39",
  "QBCSCORE", "Composite Score",                                                  NA_character_, "Composite Score",                     "VFQ-25",
  "QOCSCORE", "Composite Score (incl. Optional Items)",                           NA_character_, "Composite Score",                     "VFQ-39"
) %>%
  mutate(
    PARCAT1 = "VFQ-25 INTERVIEWER ADMINISTERED",
    PARCAT2 = "Derived Scale"
  ) %>%
  select(PARAM, PARAMCD, PARCAT1, PARCAT2, PARCAT3, PARCAT4, PARCAT5)
```

#### Original Items (`param_lookup_original`)

```{r, eval=TRUE, echo=FALSE}
dataset_vignette(
  param_lookup_original,
  display_vars = exprs(QSTESTCD, PARAM, PARAMCD, PARCAT1, PARCAT2, PARCAT3, PARCAT4, PARCAT5)
)
```

#### Transformed Records (`param_lookup_transformed`)

```{r, eval=TRUE, echo=FALSE}
dataset_vignette(
  param_lookup_transformed,
  display_vars = exprs(PARAM, PARAMCD, PARCAT1, PARCAT2, PARCAT3, PARCAT4, PARCAT5)
)
```

#### Composite Records (`param_lookup_composite`)

```{r, eval=TRUE, echo=FALSE}
dataset_vignette(
  param_lookup_composite,
  display_vars = exprs(PARAM, PARAMCD, PARCAT1, PARCAT2, PARCAT3, PARCAT4, PARCAT5)
)
```
## Initial Set Up and Mapping the Raw Records {#mapping_raw} 

We can now start setting up ADVFQ by taking `qs` and merging on some ADSL variables which are needed for BDS derivations later down the line. Note that there is no need to merge on the whole ADSL dataset immediately as in so doing we would be needlessly increasing the size of the working dataset; the rest of the variables can be merged on at the end. 

```{r, eval = TRUE}
adsl_vars <- exprs(TRTSDT, TRTEDT, TRT01A, TRT01P)

advfq <- qs %>%
  derive_vars_merged(
    dataset_add = adsl,
    new_vars = adsl_vars,
    by_vars = get_admiral_option("subject_keys")
  )
```

Now we can derive the analysis date (`ADT`) and analysis relative day (`ADY`) variables. These derivations are study-specific and so the ones below are just examples - the user is again invited to consult the vignette on [creating a BDS dataset from SDTM](https://pharmaverse.github.io/admiral/articles/bds_finding.html) for details on this topic.

```{r, eval = TRUE}
advfq <- advfq %>%
  derive_vars_dt(
    new_vars_prefix = "A",
    dtc = QSDTC
  ) %>%
  derive_vars_dy(
    reference_date = TRTSDT,
    source_vars = exprs(ADT)
  )
```

Next, we can assign `PARAMCD` for the original parameters by merging with the `param_lookup_original` table we set up earlier. Using `derive_vars_merged_lookup()` allows us to get console-level feedback that all the `QSTESTCD`s have been mapped. We only add `PARAMCD` for now, and will derive `PARCATy` and `PARAM` later - again to avoid carrying forward too many variables. 

We can also derive `AVAL`, `AVALC` and `BASETYPE` for the original records with a simple `mutate()` statement, and then `AVISIT` and `AVISITN` with study-specific logic.

```{r, eval = TRUE}
advfq <- advfq %>%
  ## Add PARAMCD for original parameters only - PARCATy and PARAM will be added later
  derive_vars_merged_lookup(
    dataset_add = param_lookup_original,
    new_vars = exprs(PARAMCD),
    by_vars = exprs(QSTESTCD)
  ) %>%
  mutate(
    AVAL = QSSTRESN,
    AVALC = QSORRES,
    BASETYPE = "LAST PERIOD 01"
  ) %>%
  mutate(
    AVISIT = case_when(
      !is.na(VISIT) ~ str_to_title(VISIT),
      TRUE ~ NA_character_
    ),
    AVISITN = case_when(
      AVISIT == "Baseline" ~ 1,
      AVISIT == "Week 12" ~ 12,
      AVISIT == "Week 24" ~ 24,
      TRUE ~ NA
    ),
  )
```

Here's what the data for the "Near Activities" and "Distance Activities" questions looks like for one patient's first two visits:

```{r, eval=TRUE, echo=FALSE}
dataset_vignette(
  advfq %>%
    filter(USUBJID %in% c("01-701-1034"), PARAMCD %in% c("VFQ105", "VFQ106", "VFQ107", "VFQ108", "VFQ109", "VFQ114")) %>%
    arrange(USUBJID, AVISITN, PARAMCD),
  display_vars = exprs(USUBJID, QSTEST, AVISIT, PARAMCD, AVAL, AVALC)
)
```

## Deriving the Transformed Parameters {#deriving_transformed}

Now we are ready to derive the transformed parameters. Excluding 15C (see later down this section), all of these are derived by re-scaling a single original record. As such, the prototypical code which can be used to derive them is a call to `derive_extreme_records()` which is structured as follows:

```{r, eval = FALSE}
derive_extreme_records(
  dataset = advfq,
  dataset_add = advfq,
  filter_add = QSTESTCD == "VFQ1xx" & !is.na(AVAL),
  set_values_to = exprs(
    AVAL = transform_range(
      source = AVAL,
      source_range = c(1, 5), # or some other range, e.g. c(1, 6), depending on the parameter
      target_range = c(0, 100),
      flip_direction = TRUE # or FALSE, depending on the parameter
    ),
    PARAMCD = "QRxx"
  )
)
```

The function `transform_range()` is a helper function from `{admiral}` which performs the re-scaling. You will need to specify the `source_range` (i.e. the range of possible values for the original record) and whether or not to `flip_direction` (i.e. whether higher values of the original record correspond to better or worse outcomes). Note that both the `source_range` and `flip_direction` *will* change across parameters - for instance, the source range is `c(1, 5)` for `PARAMCD == "VFQ101"` but `c(1, 6)` for `PARAMCD == "VFQ102"`. As such it is useful to group all parameters (excluding 15C, for which a different approach will be needed) based on which transformation they require; in so doing the transformations for each group can be done together. 

```{r, eval = TRUE}
range1to5_flip_params <- c(
  "VFQ101", "VFQ103", "VFQ104", "VFQ105", "VFQ106", "VFQ107", "VFQ108",
  "VFQ109", "VFQ110", "VFQ111", "VFQ112", "VFQ113", "VFQ114", "VFQ116",
  "VFQ116A", "VFQ1A03", "VFQ1A04", "VFQ1A05", "VFQ1A06", "VFQ1A07",
  "VFQ1A08", "VFQ1A09"
)

range1to6_flip_params <- c("VFQ102")

range1to5_noflip_params <- c(
  "VFQ117", "VFQ118", "VFQ119", "VFQ120", "VFQ121", "VFQ122", "VFQ123",
  "VFQ124", "VFQ125", "VFQ1A11A", "VFQ1A11B", "VFQ1A12", "VFQ1A13"
)

range0to10_noflip_params <- c("VFQ1A01", "VFQ1A02")
```

For question 15C we have to be a little more careful as the derivation depends on whether or not question 15C was asked at that visit. If it was, then we can derive the transformed record as normal. However, if it wasn't, then we have to check the response to question 15B - if the response to 15B indicates that the patient has never driven or given up driving (i.e. `QSSTRESN == 1`), then we set the transformed record to 0; otherwise, we do not derive a transformed record for that visit. To do this, we can set up a temporary flag variable using `derive_var_merged_exist_flag()` to identify visits where question 15C was not asked, and then use that flag in the derivation of the transformed record for 15C.

```{r, eval = TRUE}
advfq <- advfq %>%
  derive_var_merged_exist_flag(
    dataset_add = advfq,
    by_vars = exprs(!!!adsl_vars, ADT, ADY),
    new_var = TEMP_VFQ115C_FL,
    condition = QSTESTCD == "VFQ115C",
    true_value = "Y",
    false_value = "N",
    missing_value = "M"
  )
```

We can then derive the transformed records, including the special handling for 15C, in a single call to `call_derivation()`, since each call to `derive_extreme_records()` contains the same assignment of `dataset`, `dataset_add` and `keep_source_vars`, then only differs in the `filter_add` and `set_values_to` arguments. Each list passed to `variable_params` performs the transformation for one of the groups of parameters set up above, including dynamical generation of the `PARAMCD`, though 15C is done separately. Note that we set `keep_source_vars = adsl_vars` to ensure SDTM records do not get populated for derived parameters.

```{r, eval = TRUE}
advfq <- advfq %>%
  call_derivation(
    derivation = derive_extreme_records,
    dataset = .,
    dataset_add = .,
    keep_source_vars = c(
      get_admiral_option("subject_keys"),
      exprs(PARAMCD, AVISIT, AVISITN, ADT, ADY),
      adsl_vars
    ),
    variable_params = list(
      params(
        filter_add = QSTESTCD %in% range1to5_flip_params & !is.na(AVAL),
        set_values_to = exprs(
          AVAL = transform_range(
            source = AVAL,
            source_range = c(1, 5),
            target_range = c(0, 100),
            flip_direction = TRUE
          ),
          PARAMCD = str_replace(QSTESTCD, "VFQ1", "QR")
        )
      ),
      params(
        filter_add = QSTESTCD %in% range1to6_flip_params & !is.na(AVAL),
        set_values_to = exprs(
          AVAL = transform_range(
            source = AVAL,
            source_range = c(1, 6),
            target_range = c(0, 100),
            flip_direction = TRUE
          ),
          PARAMCD = str_replace(QSTESTCD, "VFQ1", "QR")
        )
      ),
      params(
        filter_add = QSTESTCD %in% range1to5_noflip_params & !is.na(AVAL),
        set_values_to = exprs(
          AVAL = transform_range(
            source = AVAL,
            source_range = c(1, 5),
            target_range = c(0, 100),
            flip_direction = FALSE
          ),
          PARAMCD = str_replace(QSTESTCD, "VFQ1", "QR")
        )
      ),
      params(
        filter_add = QSTESTCD %in% range0to10_noflip_params & !is.na(AVAL),
        set_values_to = exprs(
          AVAL = transform_range(
            source = AVAL,
            source_range = c(0, 10),
            target_range = c(0, 100),
            flip_direction = FALSE
          ),
          PARAMCD = str_replace(QSTESTCD, "VFQ1", "QR")
        )
      ),
      # For QR15C, do it in two parts
      # first in the case where QSTESTCD == "VFQ115C" is present at that visit
      params(
        filter_add = QSTESTCD == "VFQ115C" & !is.na(AVAL),
        set_values_to = exprs(
          AVAL = transform_range(
            source = AVAL,
            source_range = c(1, 5),
            target_range = c(0, 100),
            flip_direction = TRUE
          ),
          PARAMCD = "QR15C"
        )
      ),
      # second in the case where QSTESTCD == "VFQ115C" is not present at that visit
      params(
        filter_add = TEMP_VFQ115C_FL == "N" & QSTESTCD == "VFQ115B" & AVAL == 1,
        set_values_to = exprs(
          AVAL = 0,
          PARAMCD = "QR15C"
        )
      )
    )
  ) %>%
  select(-TEMP_VFQ115C_FL)
```

We can then add the `PARAM` and `PARCATy` variables for both the original and transformed records by merging with the relevant lookup tables.

```{r eval = TRUE}
advfq <- advfq %>%
  derive_vars_merged_lookup(
    dataset_add = rbind(
      param_lookup_original %>% select(-QSTESTCD),
      param_lookup_transformed
    ),
    by_vars = exprs(PARAMCD)
  )
```

Here's what some of the transformed parameters and original records look like alongside each other for one patient's first two visits:

```{r, eval=TRUE, echo=FALSE}
dataset_vignette(
  advfq %>%
    filter(
      USUBJID %in% c("01-701-1034"),
      PARAMCD %in% c("VFQ101", "VFQ102", "VFQ119", "QR01", "QR02", "QR19"),
      AVISIT %in% c("Baseline", "Week 12")
    ) %>%
    arrange(USUBJID, AVISITN, PARAMCD) %>%
    select(USUBJID, AVISIT, PARAM, PARAMCD, AVAL),
  display_vars = exprs(USUBJID, AVISIT, PARAM, PARAMCD, AVAL)
)
```

## Deriving the Composite Parameters {#deriving_composite}

### Composite Scores by Category

With the now-derived transformed parameters (identifiable through `PARCAT2 == "Transformed - Original Items"`), we have every question measured on the same scale and can derive the composite parameters. These are essentially just means across categories of questions, with the category being determined by the contents of `PARCAT4`. The VFQ guidelines dictate that for each category, two different composite parameters should be derived: one including just the Base items of the questionnaire (`PARCAT5 == "Base Item"`) and the other one also including any Optional items. As such, below we derive the composite parameters in two stages, first constructing `advfq_qsb` for the Base item-only composite means, and then `advfq_qso` for the means including all items. 

Note that the `dataset` argument is not passed to `derive_summary_records()`, meaning the output datasets contain only the new composite parameters. Also, this time we use `PARCAT4` as a by-variable, meaning that the new composite scores are assigned the same value of `PARCAT4` (e.g. "Distance Activities") as the records that were used to constitute them. Then, we can then use `derive_vars_merged_lookup()` to add on the `PARCATy` (excluding `PARCAT4`) and `PARAM`/`PARAMCD` variables for these new records. At the end, we append the new records to the existing `advfq` using `bind_rows()`.

```{r, eval = TRUE}
advfq_qsb <- derive_summary_records(
  dataset_add = advfq,
  filter_add = PARCAT2 == "Transformed - Original Items" & PARCAT5 == "Base Item" & !is.na(AVAL),
  by_vars = c(
    get_admiral_option("subject_keys"),
    exprs(!!!adsl_vars, AVISIT, AVISITN, ADT, ADY, PARCAT4)
  ),
  set_values_to = exprs(AVAL = mean(AVAL))
) %>%
  derive_vars_merged_lookup(
    dataset_add = filter(param_lookup_composite, str_starts(PARAMCD, "QSB")),
    new_vars = exprs(PARAMCD, PARAM, PARCAT1, PARCAT2, PARCAT3, PARCAT5),
    by_vars = exprs(PARCAT4)
  )

advfq_qso <- derive_summary_records(
  dataset_add = advfq,
  filter_add = PARCAT2 == "Transformed - Original Items" & !is.na(AVAL),
  by_vars = c(
    get_admiral_option("subject_keys"),
    exprs(!!!adsl_vars, AVISIT, AVISITN, ADT, ADY, PARCAT4)
  ),
  set_values_to = exprs(AVAL = mean(AVAL))
) %>%
  derive_vars_merged_lookup(
    dataset_add = filter(param_lookup_composite, str_starts(PARAMCD, "QSO")),
    new_vars = exprs(PARAMCD, PARAM, PARCAT1, PARCAT2, PARCAT3, PARCAT5),
    by_vars = exprs(PARCAT4)
  )

advfq <- bind_rows(advfq, advfq_qsb, advfq_qso)
```

Here's what the Base item composite scores by category look like for the "Near Activities" and "Distance Activities" categories for one patient's first two visits:

```{r, eval=TRUE, echo=FALSE}
dataset_vignette(
  advfq %>%
    filter(USUBJID %in% c("01-701-1034"), PARAMCD %in% c("QR05", "QR06", "QR07", "QR08", "QR09", "QR14", "QSBNA", "QSBDA")) %>%
    arrange(USUBJID, AVISITN, PARAMCD) %>%
    select(USUBJID, AVISIT, PARAM, PARAMCD, PARCAT4, AVAL),
  display_vars = exprs(USUBJID, AVISIT, PARAM, PARAMCD, PARCAT4, AVAL)
)
```

### Overall Composite Scores

Finally, for the overall composite scores, we take all the newly-derived composite parameters, excluding "General Health", and create:

* An averaged record (`PARAMCD == "QBCSCORE"`) using the composite scores that were calculated with Base items only;
* An averaged record (`PARAMCD == "QOCSCORE"`) using the composite scores that were calculated with Base and Optional items.

As was done for the [transformed parameters](#deriving_transformed), we do this in a single call to `call_derivation()`, since the only difference between the two derivations is the `filter_add` and `set_values_to` arguments.

```{r, eval = TRUE}
advfq <- advfq %>%
  call_derivation(
    derivation = derive_summary_records,
    dataset_add = advfq,
    by_vars = c(
      get_admiral_option("subject_keys"),
      exprs(!!!adsl_vars, AVISIT, AVISITN, ADT, ADY)
    ),
    variable_params = list(
      params(
        # Use Base Items only
        filter_add = PARCAT5 == "VFQ-25" & str_sub(PARAMCD, 1, 3) == "QSB" &
          PARCAT4 != "General Health" & !is.na(AVAL),
        set_values_to = exprs(
          AVAL = mean(AVAL),
          PARAMCD = "QBCSCORE"
        )
      ),
      params(
        # Use optional items items only
        filter_add = PARCAT5 == "VFQ-39" & str_sub(PARAMCD, 1, 3) == "QSO" &
          PARCAT4 != "General Health" & !is.na(AVAL),
        set_values_to = exprs(
          AVAL = mean(AVAL),
          PARAMCD = "QOCSCORE"
        )
      )
    )
  )
```

We then assign `PARAM`/`PARAMCD` and `PARCATy` for these new overall composite records by merging with the relevant lookup table, and append them to `advfq`.

```{r, eval = TRUE}
advfq <- advfq %>%
  filter(str_detect(PARAMCD, "SCORE")) %>%
  select(-PARAM, -starts_with("PARCAT")) %>%
  derive_vars_merged_lookup(
    dataset_add = param_lookup_composite,
    new_vars = exprs(PARAM, PARCAT1, PARCAT2, PARCAT3, PARCAT4, PARCAT5),
    by_vars = exprs(PARAMCD)
  ) %>%
  rbind(advfq %>% filter(!str_detect(PARAMCD, "SCORE")))
```

Here's what the overall composite records using the Base and Optional items for one patient's baseline visit:

```{r, eval=TRUE, echo=FALSE}
dataset_vignette(
  advfq %>%
    filter(
      USUBJID %in% c("01-701-1034"),
      (PARCAT5 == "VFQ-39" & str_sub(PARAMCD, 1, 3) == "QSO" & PARCAT4 != "General Health") | PARAMCD == "QOCSCORE",
      AVISIT == "Baseline"
    ) %>%
    arrange(USUBJID, AVISITN, PARAMCD) %>%
    select(USUBJID, AVISIT, PARAM, PARAMCD, PARCAT4, AVAL),
  display_vars = exprs(USUBJID, AVISIT, PARAM, PARAMCD, PARCAT4, AVAL)
)
```

## Further Derivations of Standard BDS Variables {#further}

The user is invited to consult the article on [creating a BDS dataset from SDTM](https://pharmaverse.github.io/admiral/articles/bds_finding.html) to learn how to add standard BDS variables to ADVFQ. 

## Example Script {#example}

ADaM | Sample Code 
---- | -------------- 
ADVFQ | [ad_advfq.R](https://github.com/pharmaverse/admiralophtha/blob/main/inst/templates/ad_advfq.R)
