Motivation
The "foreign" package for R already provides facilities to
import data from other statistical software packages such as SPSS or
Stata, but they are limited by the way survey data are generally
represented in R. That is, since variables in an R data frame
can only be numerical vectors or factors, any direct translation of SPSS
or Stata data sets into data frames will lead to the loss of information
of information, such as variable labels, variable labels, or
user-specified missing values. (Value labels can be preserved by
translating them into factor levels, but this means losing information
about the original codes. It will also lead to undesired missing values,
if variables in the original data sets are only partially labelled.) The
"memisc" package for this reason provides functions that allow to import
SPSS or Stata data sets into objects of the class
"data.set" defined in it.
The role of "importer" objects
Importing data using the facilities provided by the "memisc" package
consists of two steps. In the first step, a description of the data in
the file is collected in an object of class "importer". In the second
step, data are imported into "data.set" objects with the help of these
"importer" objects. These "importer" objects contain only meta-data
e.g. about variable labels, value labels, and user-defined missing
values. This allows to get an overview of the structure of the file
without the need of loading the complete data, which is advantageous
esp. if the data set is large. For example, with the help of an
"importer object" it is possible to see what the labels of the variables
are so that one can select those variables from the data file that are
actually needed. The data set object in R memory can then created by --
if imprtr is an importer object -- by calls like
subset(imprtr,...), imprtr[...] or
as.data.set(imprtr). Some examples are given in the
following.
Note that these examples require data not included in the package (you need to register to GESIS to download the data). The vignette code cannot be run without this additional data.
Importing data from SPSS files
Importing from a SPSS "system" file
In order to import data from an SPSS "system" file, the usual binary format in which SPSS data now is usually saved and often distributed, one needs to first make the file that contains the data known to R, as in the following example:
library(memisc)
ZA5702 <- spss.system.file("Data/ZA5702_v2-0-0.sav")
ZA5702:
SPSS system file 'Data/ZA5702_v2-0-0.sav'
with 979 variables and 3911 observationsOnce the "system file" is declared using the function
spss.system.file(), metadata becomes available, such as the
number of cases and variables (as just seen), the names and labels of
the variables (as seen below):
description(ZA5702):
study 'Studiennummer'
version 'GESIS Archiv Version'
year 'Erhebungsjahr'
field 'Erhebungszeitraum'
glescomp 'GLES-Komponente'
survey 'Erhebung/Welle'
lfdn 'Laufende Nummer (Kumulation)'
vlfdn 'Laufende Nummer (Vorwahl)'
nlfdn 'Laufende Nummer (Nachwahl)'
datum 'Datum der Befragung (Monat/Tag/Jahr)'(Here only an extract of the full output was shown, since the data set contains as many as 979 variables.)
An "importer" object, such as ZA5702 in this example,
would also allow to obtain a full codebook with
codebook(ZA5702)but we refrain from showing such a codebook for the obvious reason of
not creating too much output. As the inspection of the data in the file
shows, most variable names have a standardised, yet non-mnemonic
structure. Variables referring to questions asked in the pre-election
wave of the GLES 2013 study have names starting with "v",
those referring to questions asked in the post-election wave have names
starting with "v", while those referring to question asked
in both waves have names starting "nv". For a specific
analysis, such variable names are not very useful. For this reason we
want to rename them. We could do this after loading the data, but it is
more convenient to do the data import and the renaming in one step as in
the example below:
gles2013work <- subset(ZA5702,
select=c(
wave = survey,
intent.turnout = v10,
turnout = n10,
voteint.candidate = v11aa,
voteint.list = v11ba,
postal.vote.candidate = v12aa,
postal.vote.list = v12ba,
vote.candidate = n11aa,
vote.list = n11ba,
bula = bl
))The variable names to the left of the equality sign are the variable names as they will appear in the data set after import, while the variable names to the right of the equality aign are the variable names as they exist in the data file.
As a demonstration of what information can be extracted from the data file, we create a codebook for one of the items in the data set:
codebook(gles2013work$turnout)================================================================================
gles2013work$turnout 'Wahlbeteiligung'
--------------------------------------------------------------------------------
Storage mode: double
Measurement: interval
Missing values: -Inf - -1
Values and labels N Valid Total
-99 M 'keine Angabe' 3 0.1
-97 M 'trifft nicht zu' 20 0.5
-94 M 'nicht in Auswahlgesamtheit' 2003 51.2
1 'ja, habe gewaehlt' 1596 84.7 40.8
2 'nein, habe nicht gewaehlt' 289 15.3 7.4
Min: 1.000
Max: 2.000
Mean: 1.153
Std.Dev.: 0.360 Import from a SPSS "portable" file
Data from SPSS "portable" files are imported in essentially the same way as data from SPSS "system" files: The first step again is to make the data set known to R:
ZA3861 <- spss.portable.file("Data/ZA3861.por",iconv=FALSE)
ZA3861:
SPSS portable file 'Data/ZA3861.por'
with 331 variables and 3263 observationsSince this file contains German umlauts (in contrast to the previous example), we need to convert the character coding of the value labels etc. from "Latin-1" (the original coding of the data) into the native encoding of the system (unless the computer is using natively "Latin-1" encoding and not - as must Mac and most Linux System - a variant of UTF8).
ZA3861 <- Iconv(ZA3861,from="latin1")Importer objects created from "portable" files can be examined in the same way as importer objects created from "system" files. For example, we get a description of the variables in the data set (the variable labels) and a codebook.
description(ZA3861):
vvpnid 'Fallnummer'
vsplitwo 'West-Ost-Kennung'
vvornach 'Vor-/Nachwahl'
vland 'Bundesland'
v10 'Wirtschaftl. Lage allgemein'
v20 'Wirtschaftl. Lage retrospektiv'
v30 'Wirtschaftl. Lage prospektiv'
v31 'Wichtigkeit Erst/Zweitstimme BTW (nicht 94)'
v40 'Demokratiezufriedenheit'
v50 'Staerke Politikinteresse' To actually import the data and make them accessible for analysis we
can (as above), use as.data.set(), or subset()
as in this example:
work2002 <- subset(ZA3861,
select=c(
respid = VVPNID,
split.wo = VSPLITWO,
split.vor.nach = VVORNACH,
Bundesland = VLAND,
Erststimme = V69,
Zweitstimme = V70,
Geschlecht = VSEX,
GebMonat = VMONAT,
GebJahr = VJAHR,
Konfession = VRELIG,
Kirchgang = VKIRCHG,
Erwerbst = VBERUFTG,
FrErwerbst = VFRBERTG,
Beruf = VBERUF,
Famstand = VFAMSTDN,
Partner = VPARTNER,
BildungP = VPBILDGA,
BerufstP = VPBERUFT,
FrBerufstP = VPFBERTG,
BerufP = VPBERUF,
ReprGewicht = VGVWNW
)
)Import from a fixed-width file accompanied by SPSS syntax
Data from more recent study components of the American National
Elecion Study comes in fixed-width format, with some additional SPSS
syntax files that define columns, variable labels, value labels, and
missing values. memisc also provides an importer function
such data. Naturally this requires a little bit more information. In
addition to the actual data file, we also need a file with SPSS syntax
specifying the data columns. Optionally, Syntax files that define
variable labels, value lables, and missing values can also be
specified.
anes2008TS <- spss.fixed.file("Data/anes2008/anes2008TS_dat.txt",
columns.file="Data/anes2008/anes2008TS_col.sps",
varlab.file="Data/anes2008/anes2008TS_lab.sps",
codes.file="Data/anes2008/anes2008TS_cod.sps",
missval.file="Data/anes2008/anes2008TS_md.sps")
anes2008TS:
SPSS fixed column file 'issues/anes2008/anes2008TS_dat.txt'
with 1954 variables and 2322 observations
with variable labels from file 'issues/anes2008/anes2008TS_lab.sps'
with value labels from file 'issues/anes2008/anes2008TS_cod.sps'
with missing value definitions from file 'issues/anes2008/anes2008TS_md.sps' Further information about the data can now be obtained from the
returned importer object in the same way as from importer objects that
describe SPSS "system" or SPSS "portable" files. That is, we can use
names(), description(), and
codebook(). To get the data in to the memory of R
we can use (as above) the functions as.data.set() and
subset().
Importing data from a Stata file
Data from Stata files (up to Stata Version 12) can be imported in the same way as data from SPSS files. The main difference is the function used for it, and the fact that user-defined missing values do not exists in Stata. For this, see the following example:
library(memisc)
ZA5702.dta <- Stata.file("Data/ZA5702_v2-0-0.dta")
ZA5702.dta:
Stata file 'Data/ZA5702_v2-0-0.dta'
with 874 variables and 3911 observations
gles2013work.dta <- subset(ZA5702.dta,
select=c(
wave = survey,
intent.turnout = v10,
turnout = n10,
voteint.candidate = v11aa,
voteint.list = v11ba,
postal.vote.candidate = v12aa,
postal.vote.list = v12ba,
vote.candidate = n11aa,
vote.list = n11ba,
bula = bl
))
codebook(gles2013work.dta$turnout)================================================================================
gles2013work.dta$turnout 'Wahlbeteiligung'
--------------------------------------------------------------------------------
Storage mode: integer
Measurement: nominal
Missing values: 100 - 127
Values and labels N Percent
-99 'keine Angabe' 3 0.1
-98 'weiss nicht' 0 0.0
-97 'trifft nicht zu' 20 0.5
-96 'Split' 0 0.0
-95 'nicht teilgenommen' 0 0.0
-94 'nicht in Auswahlgesamtheit' 2003 51.2
-93 'Interview abgebrochen' 0 0.0
-92 'Fehler in Daten' 0 0.0
-86 'nicht wahlberechtigt' 0 0.0
-85 'nicht waehlen' 0 0.0
-84 'keine Erst-/Zweitstimme abgegeben' 0 0.0
-83 'ungueltig waehlen' 0 0.0
-82 'keine andere Partei waehlen' 0 0.0
-81 'noch nicht entschieden' 0 0.0
-72 'nicht einzuschaetzen' 0 0.0
-71 'nicht bekannt' 0 0.0
1 'ja, habe gewaehlt' 1596 40.8
2 'nein, habe nicht gewaehlt' 289 7.4