formr Runs

---

A formr "run" contains your study's complete design. Designs can range from the simple (a single survey or a randomised experiment) to the complex (like a diary study with daily reminders by email and text message or a longitudinal study tracking social network changes).

Module explanations

Survey Spreadsheet

---

You can clone a Google spreadsheet to get started or start with an empty spread sheet.

Some helpful tips:

• You may want to make linebreaks in Excel to format your text. In Microsoft Excel on Macs, you need to press Command ⌘+Option ⌥+Enter ↩, on Windows it is ctrl+Enter ↩. We suggest you start working from the provided sample sheet, because it already has the proper formatting and settings. In Google Spreadsheets, the combination is Option ⌥+Enter ↩.
• Make text bold using __bold__, make it italic using *italic*.

type	name	label	optional	showif
text	name	Please enter your name	*
number 1,130,1	age	How old are you?
mc agreement	emotional_stability1R	I worry a lot.		age >= 18
mc agreement	emotional_stability2R	I easily get nervous and unsure of myself.		age >= 18
mc agreement	emotional_stability3	I am relaxed and not easily stressed.		age >= 18

Available columns

You can use more columns than the ones shown above. Unknown column types are simply ignored, so you can use them for other information.

The following column types exist:

type

set the item type (see item types tab)

name

this is simply the name of the item. You'll use it to refer to the item in your data analysis and when making complex conditions, so adhere to a systematic naming scheme (we recommend scale1, scale2, scale3R for Likert-type items).

label

This column is the text that will be shown on the left-hand side of the answer choices (for most items). You can use Markdown formatting here.

showif

If you leave this empty, the item will always be shown. If it contains a condition, such as sex == 1, it will only be shown if that condition is true. Conditions are written in R and can be arbitrarily complex. You should always test them well. It is also possible to refer to data in other surveys using e.g. other_survey$item_name != 2. If you refer to data on the same page, items will also be shown dynamically using Javascript.

optional

Nearly all items are mandatory by default. By using * in this column, you can turns items optional instead. Using ! requires a response to items that are optional by default (check, check_button).

value

Sometimes you may want a value to be pre-set when users first fill out a form. This can be especially handy in longitudinal studies, where you want to check that e.g. contact information is still up-to-date or when you want to highlight changes across days. You can, again, use arbitrarily complex R code (e.g. a_different_survey$item1 + a_different_survey$item2) to pre-set a value, but you can also simply use 1 to choose the option with the value 1 (remember that choices for mc-family items are saved as numbers, not as the choice labels per default). There is one special word, sticky, which always pre-sets the items value to the most recently chosen value. You also have to keep in mind when pre-setting strings, that they have to be marked up in R, like this "I am text" (preferably do not use single quotes because Excel will mess them up).

class

This column can optionally be added to visually style items. Find the available classes below.

Optional classes for visual styling

You might want to tinker with the look of certain form items. To do so you can use a variety of pre-set CSS classes. This is a fancy way of saying that if you make a new column in your survey sheet, call it "class" and add space-separated magic words, stuff will look different.

These are the available styling classes:

left100, (200, …, 900)

controls the width of the left-hand column (labels). The default is left300, you have 100 pixel increments to choose from.

right100, (200, …, 900)

controls the width of the right-hand column (answers). There is no default here, usually the right-hand column will extend in accordance with the display width.

right_offset0, (100, …, 900)

controls the offset (distance) of the right-hand column to the left (not the label column, just the left). This is 300 pixels (+20 extra) by default. Analogously with left_offset100 etc. (defaults to 0).

label_align_left
label_align_center
label_align_right

controls the text alignment of the left-hand column (labels), by default it is aligned to the right.

answer_align_left
answer_align_center
answer_align_right

controls the text alignment of the right-hand column (answers), by default it is aligned to the left.

answer_below_label

This leads to answers stacking below labels, instead of them being side-by-side (the default). It entails zero offsets and left alignment. Can be overridden with offsets and the alignment classes.

hide_label

This hides the labels for mc and mc_multiple replies. Useful in combination with a fixed width for mc, mc_multiple labels and mc_heading – this way you can achieve a tabular layout. On small screens labels will automatically be displayed again, because the tabular layout cannot be maintained then.

show_value_instead_of_label

This hides the labels for mc_button and mc_multiple_button, instead it shows their values (useful numbers from 1 to x). Useful in combination with mc_heading – this way you can achieve a tabular layout. On small screens labels will automatically be displayed again, because the tabular layout cannot be maintained then.

rotate_label45, rotate_label30,
rotate_label90

This rotates the labels for mc and mc_multiple replies. Useful if some have long words, that would lead to exaggerated widths for one answer column.

mc_block

This turns answer labels for mc-family items into blocks, so that lines break before and after the label.

mc_vertical

This makes answer labels for mc-family items stack up. Useful if you have so many options, that they jut out of the viewport. If you have very long list, consider using select-type items instead (they come with a search function).

mc_horizontal

This makes answer labels for mc-family items stay horizontal on small screens.

mc_equal_widths

This makes answer labels for mc-family items have equal widths, even though their contents would lead them to have different widths. This won't work in combination with every other option for mc-styling and if your widest elements are very wide, the choices might jut out of the viewport.

mc_width50 (60, … ,
100, 150, 200)

This makes choice labels and choice buttons for mc-family items have fixed widths. If one choice has text wider than that width, it might jut out or ignore the fixed width, depending on the browser.

rating_button_label_width50
(60, … , 100, 150, 200)

This makes the labels for rating_button items have fixed widths. This can be useful to align rating_buttons buttons with each other even though the end points are labelled differently. A more flexible solution would be to horizontally center the choices using answer_align_center.

space_bottom_10
(10, 20, … , 60)

Controls the space after an item. Default value is 15.

space_label_answer_vertical_10
(10, 20, … , 60)

Controls the vertical space between label and choices, if you've set answer_below_label. Default value is 15.

clickable_map

If you use this class for a text type item, with one image in the label, this image will become a clickable image, with the four outer corners selectable (the selection will be stored in the text field). Will probably require customisation for your purposes.

Choices Spreadsheet

---

You can clone a Google spreadsheet to get started.

list_name	name	label
agreement	1	disagree completely
agreement	2	rather disagree
agreement	3	neither agree nor disagree
agreement	4	rather agree
agreement	5	agree completely

Survey Item Types

---

There are a lot of item types, in the beginning you will probably only need a few though. To see them in action, try using the following Google spreadsheet or fill it out yourself. It contains example uses of nearly every item there is.

Plain display types

note

display text. Notes are only displayed once, you can think of them as being "answered" simple by submitting.

note_iframe

If you want to render complex rmarkdown htmlwidgets, use this.

submit timeout

display a submit button. No items are displayed after the submit button, until all of the ones preceding it have been answered. This is useful for pagination and to ensure that answers required for showif or for dynamically generating item text have been given. If you specify the optional timeout (an integer, milliseconds), the submit button will automatically submit after that time has passed. However, if not all items are answered or optional, the user will end up on the same page. Together with optional items, this is a way to use timed submissions. The data in the item display table can be used to check how long an item was displayed and whether this matches with the server's time for when it sent the item and received the response.

Simple input family

text max_length

allows you to enter a text in a single-line input field. Adding a number text 100 defines the maximum number of characters that may be entered.

textarea max_length

displays a multi-line input field

number min, max, step

for numbers. step defaults to 1, using any will allow any decimals.

letters max_length

like text, allows only letters (A-Za-züäöß.;,!: ), no numbers.

email

for email addresses. They will be validated for syntax, but they won't be verified unless you say so in the run.

Sliders

range min,max,step

these are sliders. The numeric value chosen is not displayed. Text to be shown to the left and right of the slider can be defined using the choice1 and choice2 fields. Defaults are 1,100,1.

range_ticks min,max,step

like range but the individual steps are visually indicated using ticks and the chosen number is shown to the right.

Datetime family

date min,max

for dates (displays a date picker). Input can be constrained using the min,max parameters. Allowed values would e.g. be 2013-01-01,2014-01-01 or -2years,now.

time min,max

for times (displays an input with hours and minutes). Input can also be constrained using min,max, e.g. 12:00,17:00

Fancy family

geopoint

displays a button next to a text field. If you press the button (which has the location icon on it) and agree to share your location, the GPS coordinates will be saved. If you deny access or if GPS positioning fails, you can enter a location manually.

color

allows you to pick a color, using the operating system color picker (or one polyfilled by Webshims)

Multiple choice family

The, by far, biggest family of items. Please note, that there is some variability in how the answers are stored. You need to know about this, if you (a) intend to analyse the data in a certain way, for example you will want to store numbers for Likert scale choices, but text for timezones and cities (b) if you plan to use conditions in the run or in showif or somewhere else where R is executed. (b) is especially important, because you might not notice if demographics$sex == 'male' never turns true because sex is stored as 0/1 and you're testing as female.

mc choice_list

multipe choice (radio buttons), you can choose only one.

mc_button choice_list

like mc but instead of the text appearing next to a small button, a big button contains each choice label

mc_multiple choice_list

multiple multiple choice (check boxes), you can choose several. Choices defined as above.

mc_multiple_button

like mc_multiple and mc_button

check

a single check box for confirmation of a statement.

check_button

a bigger button to check.

rating_button
min, max, step

This shows the choice1 label to the left, the choice2 label to the right and a series of numbered buttons as defined by min,max,step in between. Defaults to 1,5,1.

sex

shorthand for mc_button with the ♂, ♀ symbols as choices

select_one choice_list

a dropdown, you can choose only one

select_multiple choice_list

a list in which, you can choose several options

select_or_add_one
choice_list, maxType

like select_one, but it allows users to choose an option not given. Uses Select2. maxType can be used to set an upper limit on the length of the user-added option. Defaults to 255.

select_or_add_multiple
choice_list, maxType,
maxChoose

like select_multiple and select_or_add_one, allows users to add options not given. maxChoose can be used to place an upper limit on the number of chooseable options.

mc_heading choice_list

This type permits you to show the labels for mc or mc_multiple choices only once.
To get the necessary tabular look, assign a constant width to the choices (using e.g. mc-width100), give the heading the same choices as the mcs, and give the following mcs (or mc_multiples) the same classes + hide_label.
On small screens the mc_heading will be hidden and labels will automatically be displayed again, because the tabular layout would otherwise break down.

Hidden family

These items don't require the user to do anything, so including them simply means that the relevant value will be stored. If you have exclusively hidden items in a form, things will wrap up immediately and move to the next element in the run. This can be useful for hooking up with other software which sends data over the query string i.e. https://formr.org/run_name?param1=10&user_id=29

calculate

in the value column you can specify an R expression, the result of which will be saved into this variable. Useful to pull in external data or to forestall recalculating something repeatedly that you want to refer to later. If the calculation is based on values from the same module, you can insert the calculate item in the last line of the sheet behind the last submit button and its result will be stored in the database for use in further modules.

ip

saves your IP address. You should probably not do this covertly but explicitly announce it.

referrer

saves the last outside referrer (if any), ie. from which website you came to formr

server var

saves the $_SERVER value with the index given by var. Can be used to store one of 'HTTP_USER_AGENT', 'HTTP_ACCEPT', 'HTTP_ACCEPT_CHARSET', 'HTTP_ACCEPT_ENCODING', 'HTTP_ACCEPT_LANGUAGE', 'HTTP_CONNECTION', 'HTTP_HOST', 'QUERY_STRING', 'REQUEST_TIME', 'REQUEST_TIME_FLOAT'. In English: the browser, some stuff about browser language information, some server stuff, and access time.

get var

saves the var from the query string, so in the example above get param1 would lead to 10 being stored.

random min,max

generates a random number for later use (e.g. randomisation in experiments). Minimum and maximum default to 0 and 1 respectively. If you specify them, you have to specify both.

hidden

you can use this item with a pre-set value, if you need to use data from previous pages together with data on the same page for a showif

block

Blocks progress. You can give this item a showif such as (item1 + item2) > 100 to add further requirements.

Knit R & Markdown

---

This section gives some guidance on how to format and customise text in formr. In many cases you'll do it right by default.

Markdown

You can format text/feedback everywhere (i.e. item labels, choice labels, the feedback shown in pauses, stops, in emails) in a natural fashion using Github-flavoured Markdown.
The philosophy is that you write like you would in a plain-text email and Markdown turns it nice.
In most cases, characters with special meaning won't entail unintended side effects if you use them normally, but if you ever need to specify that they shouldn't have side effects, escape it with a backslash: \*10\* doesn't turn italic.

* list item 1
* list item 2

will turn into a nice bulleted list.

• list item 1
• list item 2

# at the beginning of a line turns it into a large headline, ## up to ###### turn it into smaller ones.

*italics* and __bold__ are also easy to do.

[Named links](https://yihui.name/knitr/) and embedded images ![image description](https://imgur.com/imagelink) are easy. If you simply paste a link, it will be clickable automatically too, even easier. Email addresses are a bit special, you need the "mailto:" prefix: [Contact us](mailto:contact_email@example.com).

You can quote something by placing a > at the beginning of the line.

If you're already familiar with HTML you can also use that instead, though it is a little less readable for humans. Or mix it with Markdown! You may for example use it to go beyond Markdown's features and e.g. add icons to your text using <i class="fa fa-smile-o"></i> to get for instance. Check the full set of available icons at Font Awesome.

Knitr

If you want to customise the text or generate custom feedback, including plots, you can use Knitr. Thanks to Knitr you can freely mix Markdown and chunks of R. Some examples:

• Today is `r date()` shows today's date.
  
• Hello `r demographics$name` greets someone using the variable "name" from the survey "demographics".
  
• Dear `r ifelse(demographics$sex == 1, 'Sir', 'Madam')` greets someone differently based on the variable "sex" from the survey "demographics".
  
• You can also plot someone's extraversion on the standard normal distribution.

  ```{r}
  library(formr)
  # build scales automatically
  big5 = formr_aggregate(results = big5)
  # standardise
  big5$extraversion = scale(big5$extraversion, center = 3.2, scale = 2.1)
  
  # plot
  qplot_on_normal(big$extraversion, xlab = "Extraversion")
  ```
          

  yields
  

R Helpers

---

Wherever you use R in formr you can also use the functions in its R package. If you want to use the package in a different environment, you'll need to install it using these two lines of code.

install.packages("devtools")
devtools::install_github("rubenarslan/formr")

The package currently has the following feature sets

• Some shorthand functions for frequently needed operations on the site:

  first(cars); 
  last(cars); 
  current(cars); 
  "formr." %contains% "mr."

• Some helper functions to make it easier to correctly deal with dates and times:

  time_passed(hours = 7); 
  next_day(); 
  in_time_window(time1, time2);

• Connecting to formr, importing your data, correctly typing all variables, automatically aggregating scales.
• Easily making feedback plots e.g.

  qplot_on_normal(0.8, "Extraversion")

  The package also has a function to simulate possible data, so you can try to make feedback plots ahead of collecting data.

Features

---

The following designs and many more are possible:

• simple surveys with and without feedback
• complex surveys (using skipping logic, personalised text, complex feedback)
• surveys with eligibility limitations
• diary studies including completely flexible automated email/text message reminders
• longitudinal studies (e.g. automatically re-contact participants after they return from their exchange year). The items of later waves need not exist in final form at wave 1.
• longitudinal social networks and other studies that require rating a variable number of things or persons

Core strengths

• generates very pretty feedback live, including ggplot2, and interactive ggvis plots and htmlwidgets. We find that this greatly increases interest and retention in our studies.
• automates complex experience sampling, diary and training studies, including automated reminders via email or text message
• looks nice on a phone (about 30-40% of participants fill out our surveys on a mobile device)
• easily share, swap and combine surveys (they're simply spreadsheets) and runs (you can share complete designs, e.g. "daily diary study")
• you can use R to do basically anything that R can do (i.e. complicated stuff, like using a sentiment analysis of a participant's Twitter feed to decide when the survey happens)
• not jealous at all – feel free to integrate other components (other survey engines, reaction time tasks, whatever you are used to) with formr, we tried our best to make it easy.

Features

• manage access to and eligibility for studies
• longitudinal studies
• send text messages (see the HowTo)
• works on all somewhat modern devices and degrades gracefully where it doesn't
• formats text using Github-flavoured Markdown (a.k.a. the easiest and least bothersome way to mark up text)
• file, image, video, sound uploads for users (as survey items) and admins (to supply study materials)
• complex conditional items
• a dedicated formr R package: makes pretty feedback graphs and complex run logic even simpler. Simplifies data wrangling (importing, aggregating, simulating data from surveys).
• a nice editor, Ace, for editing Markdown & R in runs.

Plans:

• work offline on mobile phones and other devices with intermittent internet access (in the meantime enketo is pretty good and free too, but geared towards humanitarian aid)
• a better API (some basics are there)
• social networks, round robin studies - at the moment they can be implemented, but are a bit bothersome at first. There is a dedicated module already which might also get released as open source if there's time.
• more planned enhancements on Github

formr API

---

The formr API is the primary way to get data/results out of the platform. It's a low-level HTTP-based API that you can use principally to get results of a study for specified participants (sessions)

Resource requests to the formr API require that you have a valid access token which you can obtain by providing API credentials (a client id and a client secret)

API base URL:
https://api.formr.org

Obtaining Client ID and Client Secret

Access to the API is restricted, so only the administrators of formr are able to provide API credentials to formr users. To obtain API credentials, send an email to Cyril and your credentials will be sent to you.

Obtaining An Access Token

An access token is an opaque string that identifies a formr user and can be used to make API calls without further authentication. formr API access tokens are short-lived and have a life span of about an hour.

To generate an access token you need to make an HTTP POST request to the token endpoint of the API

POST /oauth/access_token?
     client_id={client-id}
    &client_secret={client-secret}
    &grant_type=client_credentials

This call will return a JSON object containing an access token which can be used to access the API without further authentication required.
Sample successful response

{
	"access_token":"XXXXXX3635f0dc13d563504b4d",
	"expires_in":3600,
	"token_type":"Bearer",
	"scope":null
}

The attribute expires_in indicates the number of seconds for which is token is valid from its time of creation. If there is an error for example if the client details are not correct, then an error object is returned containing an error_description and sometimes an error_uri where you can read more about the generated error. An example of an error object is

{
	"error":"invalid_client",
	"error_description":"The client credentials are invalid"
}

Making Resource Requests using generated access token

With the generated access token, you are able to make requests to the resource endpoints of the formr API. For now only the results resource endpoint has been implemented

Getting study results over the API

REQUEST

To obtain the results of a particular set of sessions in a particular run, send a GET HTTP request to the get endpoint along side the access_token obtained above together with the necessary parameters as shown below:

GET /get/results?
     access_token={access-token}
    &run[name]={name of the run as it appears on formr}
    &run[sessions]={comma separated list of session codes OR leave empty to get all sessions}
    &surveys[survey_name1]={comma separated list items to get from survey_name1 OR leave empty to get all items}
    &surveys[survey_name2]={comma separated list items to get from survey_name2 OR leave empty to get all items}

Notes:

• survey_name1 and survey_name2 should be the actual survey names
• If you want to get results for all surveys in the run you can omit the survey parameter
• If you want to get all items from a survey, keep the items list empty.

RESPONSE

The response to a results request is a JSON object. The keys of this JSON structure are the names of the survey that were indicated in the requested and the value associated to each survey entry is an array of objects representing the results collected for that survey for all the requested sessions. An example of a response object could be the following:

{
	"survey_1": [{
		"session": "sdaswew434df",
		"survey_1_item_1": "answer1",
		"survey_1_item_2": "answer2",
		"survey_1_item_2": "answer3",
	},
	{
		"session": "fdgdfg4323",
		"survey_1_item_1": "answer4",
		"survey_1_item_2": "answer5",
		"survey_1_item_2": "answer6",
	}],
	"survey_2": [........]
}

Using the formr API in R

# load the httr package
library(httr)

# Login using your client ID and client Secret to get an access token
login <- list( # define login credentials
  client_id = "bb472xxxxxxxxe1918",
  client_secret = "zEfgeyJ0eXXXXXEwYwNGZj",
  grant_type = "client_credentials"
)
request <- POST( # send POST request
  "https://api.formr.org/oauth/access_token",
  body = login, 
  encode = "form"
)
# parse response to get access token
# If there an error then the response object would contain the details of the error in response$error
response <- content(request)
access_token <- response$access_token

#With a valid access token, call API to get the results of a particular study (run)
query <- list(
  access_token = access_token,
  "run[name]" = "/enter run name here/",
  "run[session]" = "/here you can specify a full session code or just a substring(but include the beginning /",
  "surveys[survey_1]" = "item_1, item_2, item_etc.", # comma separated list of items to get from the survey or leave empty string to get all items
  "surveys[survey_2]" = "item_3, item_4, item_etc."
)
request <- GET("https://api.formr.org/get/results", query=query)
results <- content(request)

# With a valid response you can, for example, extract the results of a particular survey say 'survey_1':
survey_1 = results$survey_1
survey_1[, c("item_1","item_2")]

Help

---

Where to get help

If you're a participant in one of the studies implemented in formr, please reach out to the person running the study.

If you're running a study yourself, there's several places to look.

• this documentation is a good start, just click on any of the tabs above.
• There is a Wiki on Github. You can find a number of HowTos there and contribute yourself.
  • start with learning to create a simple survey
  • then learn to add feedback
  • and if you're up for more, discover diary studies, peer rating studies, and find out how to send texts.
• You'll find answers to some frequently asked questions there too.
• You can ask and answer questions on our mailing list.
• If you find a bug, this is the place to describe it (preferably in a way that allows us to reproduce it, but we're also accepting Yeti reports).