Manipulating and processing data in R

1	> iris[1:5, ]

Sample() command in R

As we have seen, samples are created from data for analysis. To create samples, sample() command is used and the number of samples to be drawn are mentioned.

For example, to create a sample of 10 simulation of a die, below command is used:

It gives output as:

Sample() should always produce random values. But it does not happen with the test code sometimes. If substituted with a seed value, the sample() command always produces random samples.

Seed value is the starting point for any random number generator formula. Seed value defines both, the initialization of the random number generator along with the path that the formula will follow.

Let us see how seed value is used.

This gives output as below:

Applications of Subsetting Data

Let us now see few applications of subsetting data in R:

a. Duplicate data can be removed during analysis using duplicated()function in R

Below command shows how to find duplicate data in subsets: Duplicated() function finds duplicate values and returns a logical vector that tells you whether the specific value is a duplicate of a previous value.

This gives output as below:

For all those values which are duplicate in the sample, true is returned.

b. Missing data can be identified using complete.cases() function in R

If during analysis, any row with missing data can be identified and removed as below:

complete.cases() command in R is used to find rows which are complete. It gives logical vector with the value TRUE for rows that are complete, and FALSE for rows that have some NA values.

Rows which have NA values can be removed using na.omit() function as below:

> row_name <- na.omit(file_name)

Adding Calculated Fields to Data

After you have created the appropriate subset of your data, the next step in your analysis is to perform some calculations. R makes it easy to perform calculations on columns of a data frame because each column is itself a vector.

Let us see data manipulation with R with the help of an example:

Let us see how to calculate the ratio between the lengths and width of the sepals

Command for the same is:

//Command to display the first five elements of the result

It gives the output as:

Let us discuss some variations of the operations performed on data frames in R.

a. with() function in R

To reduce the amount of typing and make code more readable, with() command is used as below:

This gives output same as above but reduced the task of typing.

b. within() function in R

Let us now see the use of within function for same task:

With() function  allows you to refer to columns inside a data frame without explicitly using the dollar

sign or even the name of the data frame itself.

With and Within can be used interchangeably.

Creating Subgroups or Bins of Data

Most statisticians often draw histograms to investigate their data. As this type of calculation is common when you use statistics, R has some functions for it.

a. Cut() function in R

Cut() function groups values of a variable into larger bins. It creates bins of equal size and classifies each element into its appropriate bin.

Let us see how cut works in R with example:

This gives the result as a factor with three levels.

The cut() function creates mathematical labels for the bins. The label names can be provided by the user.

Let us see this with the help of example:

The result shows three labels in the output.

b. table() function in R

To count the number of observations in each level of factor, R table() command can be used as below:

The result shows the output as a table containing the number of elements in each factor.

Combining and Merging Datasets in R

If you want to combine data from different sources in R, you can combine different sets of data in three ways:

a. By Adding Columns using cbind() in R

If the two sets of data have an equal set of rows, and the order of the rows is identical, then adding columns makes sense. This can be done by using the data.frame or cbind() function.

b. By Adding Rows using rbind() function in R

If both sets of data have the same columns and you want to add rows to the bottom, use rbind().

c. By Combining Data With Different Shapes using merge() function in R

The merge() function combines data based on common columns as well as rows. In database language, this is usually called joining data.

For merging the existing data, using the merge()function is useful. You can use merge()to combine data only when certain matching conditions are satisfied.

Merge() Function in R

Let us see the use of merge() function.

The merge() function is used to combine data frames. Let us see this with an example:

This is the command to create a data frame that consists of cold as well as large states.

Let us see different types of merge().

The merge() function allows four ways of combining data:

a. Natural join in R

To keep only rows that match from the data frames, specify the argument all=FALSE

b. Full outer join in R

To keep all rows from both data frames, specify all=TRUE

c. Left outer join in R

To include all the rows of your data frame x and only those from y that match, specify all.x=TRUE

d. Right outer join in R

To include all the rows of your data frame y and only those from x that match, specify all.y=TRUE

The merge()function takes a large number of arguments, as follows:

• x:A data frame
• y: A data frame
• by, by.x, by.y: Names of the columns common to both x and y. By default, it uses columns with common names between the two data frames.
• all, all.x, all.y: Logical values that specify the type of merge. The default value is all = FALSE

Match() function in R

The R match() function returns the matching positions of two vectors or, more specifically, the positions of the first matches of one vector in the second vector.

This is the command to search for large states that also occur in the data frame cold.states

It gives output as:

Sorting and Ordering Data in R using sort() in R and Order() in R

A common task in data analysis and reporting is sorting information. You can answer many everyday questions with sorted tables of data that tell you the best or worst of specific things; for example, parents want to know which school in their area is the best, and businesses need to know the most productive factories or the most lucrative sales areas.

Let us first create data frame and then we will sort it.

This is the command to create data frame some.states.

This will create subset of it.

By default, sorting is done in ascending manner if not specified.

//Command to sort Population in descending order

This is how sorting of data can be done in R.

Data frames can also be sorted as below:

Above is the command to show the order of the elements of the data frame some.states

Now to sort above data frame in ascending order, below command is used:

To sort in descending order, we need to specify as below:

This is how order() and sort() functions are used.

Traversing Data with the Apply() Function in R

To traverse the data, R uses apply functions. The output of the apply() function depends on the data structure being traversed.

a. Array or matrix

The apply() function traverses either the rows or columns of a matrix, applies a function to each resulting vector, and returns a vector of summarized results

b. List

The lapply() function can traverse a list, it applies a function to each element, and returns a list of the results. Sometimes it is possible to simplify the resulting list into a matrix or vector. lapply returns a list of the same length as X, each element of which is the result of applying FUN to the corresponding element of X.

R Apply() function is used as below:

The apply() function takes four arguments as below:

• X: This is the data—an array (or matrix)
• MARGIN: This is a numeric vector that indicates the dimension over which to traverse—1 means rows and 2 means columns
• FUN: This is the function to apply (for example, sum or mean)
• … (dots): If the FUN function requires any additional arguments, they can be added here.

In essence, the apply function allows us to make entry-by-entry changes to data frames and matrices. If MARGIN=1, the function accepts each row of X as a vector argument, and returns a vector of the results. Similarly, if MARGIN=2 the function acts on the columns of X. Most impressively, when MARGIN=c(1,2) the function is applied to every entry of X.

Let us now discuss the variations of the apply() function:

a. lapply() function in R

We have already seen it above.

b. sapply() function in R

It works on a list or vector and returns vector.

c. tapply() function in R

It is used to create tabular summaries of data. This function takes three arguments:

• X: Refers to a vector
• INDEX: Refers to a factor or list of factors
• FUN: Refers to a function

An illustrative example
Consider the code below:
#Create the matrix

# Return the product of each of the rows

# Return the sum of each of the columns

Introduction to the Formula Interface in R

The R formula interface allows you to concisely specify which columns to use when fitting a model, as well as the behavior of the model.

You need the operators when you start building models. Formula notation refers to statistical formulae, as opposed to mathematical formulae. The formula operator + means to include a column, not to mathematically add two columns together.

Operator	Example	Meaning
~	y ~ x	Model y as a function of x
+	y ~ a + b	Include columns a as well as b
–	y ~ a – b	Include a but exclude b
:	y ~ a : b	Estimate the interaction of a and b
*	y ~ a * b	Include columns as well as their interaction (that is, y ~ a + b + a:b)
|	y ~ a | b	Estimate y as a function of a conditional on b

Above table shows meanings of different operators in formula interfacing.

Variables in R

The two types of R variables are:

a. Identifier variables in R

Identifier, or ID variables identify the observations. These act as the keys that identify the observations.

b. Measured variables in R

These represent the measurements to be observed.

Getting started with reshape2 Package in R

Base R has a function, reshape() that works fine for reshaping longitudinal data.

The problem of data reshaping is far more generic than simply dealing with longitudinal data. So package reshape2 that contains several functions to convert data between long and wide format is released.

//This is the command to load reshape2 package

R reshape2 package is based on two key functions:

• Melt() in R takes wide-format data and melts it into long-format data.
• Cast() in R takes long-format data and casts it into wide-format data.