EDL
Here we provide astep-by-step overview of the core functions of the package using an example data set.
Example data
Load the example data from the package edl:
## Shape Color Category Frequency2 Frequency1
## 1 cat brown animal 88 6
## 2 rabbit brown animal 88 1
## 3 flower brown plant 6 16
## 4 tree brown plant 6 56
## 5 car brown vehicle 1 88
## 6 bicycle brown vehicle 56 32step 1: Prepare data
This data set lists all unique learning events
(i.e., the types) and their associated frequencies. However,
for a data set to function as input for the learning functions, the
dataframe must include the columns Cues and
Outcomes, and optionally Frequency. Note that
if Frequency is not included, the frequency of each
learning event is assumed to be 1.
First, we construct the columns Cues and
Outcomes for this example simulation. Here we will simulate
how two features Color and Shape may function
as cues for their category Category. We will add a
background cue “BG” to represent the learner. The different cues and
outcomes are separated using an underscore (i.e., “_“). It is possible
to another symbol, but then we will need to indicate this in the various
functions (i.e., RWlearning) with the argument
split.
dat$Cues <- paste("BG", dat$Shape, dat$Color, sep="_")
dat$Outcomes <- paste(dat$Category)
dat$Frequency <- dat$Frequency2
# remove remaining columns to simplify this example:
dat <- dat[, c("Cues", "Outcomes", "Frequency")]
# add ID for learning events:
dat$ID <- 1:nrow(dat)
head(dat)## Cues Outcomes Frequency ID
## 1 BG_cat_brown animal 88 1
## 2 BG_rabbit_brown animal 88 2
## 3 BG_flower_brown plant 6 3
## 4 BG_tree_brown plant 6 4
## 5 BG_car_brown vehicle 1 5
## 6 BG_bicycle_brown vehicle 56 6Now the data dat defines 36 unique learning events with
their associated frequencies.
##
## animal plant vehicle
## 12 12 12step 2: Create training data
The training data lists all learning events (i.e., the tokens) in their order of presentation, one learning event per row. If no column with frequencies is specificied or all frequencies are 1, the training data mirrors the data set of learning events. The training data also determines the order in which the learning events are presented to the learning network.
## Cues Outcomes Frequency ID Item Trial
## 1 BG_cat_white animal 1 13 item_13 1
## 2 BG_car_gray vehicle 1 11 item_11 2
## 3 BG_rabbit_white animal 1 14 item_14 3
## 4 BG_tree_yellow plant 1 22 item_22 4
## 5 BG_cat_brown animal 1 1 item_01 5
## 6 BG_car_blue vehicle 1 35 item_35 6## [1] 1##
## animal plant vehicle
## 398 398 398##
## 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
## 88 88 6 6 1 56 56 56 1 1 88 88 32 32 56 56 56 1 16 16 88 88 6 6 6 6
## 27 28 29 30 31 32 33 34 35 36
## 32 32 32 32 1 1 16 16 16 16Note that the function createTrainingData could also be
used to train the network on multiple (blocked or randomized) runs. We
refer to the examples in the function helpfile.
step 3: Learning
The function RWlearning trains the error-driven learning
network. The output wm is a list with weight matrices, a
weight matrix for each learning event (a learning event is basically a
row in the data frame dat). The last weight matrix shows
the connections after processing all data.
Inspection:
## [1] 1194## [1] 1194step 4: Inspection
We can now inspect the changes in the weights for after each learning event. The last weight matrix shows the connections after processing all data:
## animal
## BG 0.01
## cat 0.01
## white 0.01## animal vehicle plant
## BG 0.23192445 0.25342173 0.26004837
## cat 0.56975771 -0.25386731 -0.19252514
## white 0.02902188 0.01438611 0.08828008
## car -0.23489823 0.58349278 -0.22266026
## gray 0.08428222 0.13478297 -0.08279294
## rabbit 0.56749015 -0.25278635 -0.19893205
## tree -0.19938166 -0.20344555 0.53107493
## yellow -0.01764368 -0.05262002 0.19992315
## brown 0.18905738 0.02785816 -0.07652991
## blue -0.02279788 0.05140666 0.06043377
## red -0.02999546 0.07760785 0.07073422
## flower -0.20155237 -0.20127562 0.53348067
## bicycle -0.26949115 0.58130378 -0.19038977The function getWM retrieves the weight matrix after a
specific event, and adds zero-weight connections for the not yet
encountered cues and outcomes.
## animal vehicle plant
## BG 0.01 0 0
## cat 0.01 0 0
## white 0.01 0 0
## car 0.00 0 0
## gray 0.00 0 0
## rabbit 0.00 0 0
## tree 0.00 0 0
## yellow 0.00 0 0
## brown 0.00 0 0
## blue 0.00 0 0
## red 0.00 0 0
## flower 0.00 0 0
## bicycle 0.00 0 0We can use the functions sapply and getWM
to add zero-weight connections to all states of the network:
wm2 <- sapply(1:length(wm), function(x){getWM(wm,x)}, simplify = FALSE)
# inspect the list of states:
length(wm2)## [1] 1194## animal vehicle plant
## BG 0.01 0 0
## cat 0.01 0 0
## white 0.01 0 0
## car 0.00 0 0
## gray 0.00 0 0
## rabbit 0.00 0 0
## tree 0.00 0 0
## yellow 0.00 0 0
## brown 0.00 0 0
## blue 0.00 0 0
## red 0.00 0 0
## flower 0.00 0 0
## bicycle 0.00 0 0The functions getWeightsByCue and
getWeightsByOutcome could be used to extract the weights
per cue or per outcome.
## BG bicycle blue brown car cat flower gray rabbit red tree
## 1 0.000000 0 0.0e+00 0e+00 0.0e+00 0e+00 0 0 0 0 0.00
## 2 0.000000 0 0.0e+00 0e+00 0.0e+00 0e+00 0 0 0 0 0.00
## 3 0.000000 0 0.0e+00 0e+00 0.0e+00 0e+00 0 0 0 0 0.00
## 4 0.010000 0 0.0e+00 0e+00 0.0e+00 0e+00 0 0 0 0 0.01
## 5 0.009900 0 0.0e+00 -1e-04 0.0e+00 -1e-04 0 0 0 0 0.01
## 6 0.009801 0 -9.9e-05 -1e-04 -9.9e-05 -1e-04 0 0 0 0 0.01
## white yellow
## 1 0 0.00
## 2 0 0.00
## 3 0 0.00
## 4 0 0.01
## 5 0 0.01
## 6 0 0.01## BG bicycle blue brown car cat
## 1189 0.2597990 -0.1904500 0.06043377 -0.07659012 -0.2213647 -0.1925251
## 1190 0.2610189 -0.1904500 0.06043377 -0.07659012 -0.2213647 -0.1925251
## 1191 0.2610791 -0.1903898 0.06043377 -0.07652991 -0.2213647 -0.1925251
## 1192 0.2612898 -0.1903898 0.06043377 -0.07652991 -0.2213647 -0.1925251
## 1193 0.2613439 -0.1903898 0.06043377 -0.07652991 -0.2213647 -0.1925251
## 1194 0.2600484 -0.1903898 0.06043377 -0.07652991 -0.2226603 -0.1925251
## flower gray rabbit red tree white yellow
## 1189 0.5334265 -0.08300362 -0.1991427 0.07073422 0.5298550 0.08835573 0.1998690
## 1190 0.5334265 -0.08300362 -0.1991427 0.07073422 0.5310749 0.08957563 0.1998690
## 1191 0.5334265 -0.08300362 -0.1991427 0.07073422 0.5310749 0.08957563 0.1998690
## 1192 0.5334265 -0.08279294 -0.1989321 0.07073422 0.5310749 0.08957563 0.1998690
## 1193 0.5334807 -0.08279294 -0.1989321 0.07073422 0.5310749 0.08957563 0.1999232
## 1194 0.5334807 -0.08279294 -0.1989321 0.07073422 0.5310749 0.08828008 0.1999232## animal plant vehicle
## 1 0 0 0
## 2 0 0 0
## 3 0 0 0
## 4 0 0 0
## 5 0 0 0
## 6 0 0 0## animal plant vehicle
## 1189 -0.02999546 0.07073422 0.07760785
## 1190 -0.02999546 0.07073422 0.07760785
## 1191 -0.02999546 0.07073422 0.07760785
## 1192 -0.02999546 0.07073422 0.07760785
## 1193 -0.02999546 0.07073422 0.07760785
## 1194 -0.02999546 0.07073422 0.07760785In addition, there are various functions to extract the activations
for each learning event. The function getActivations is a
wrapper that captures most common calculations, but other, more
specialistic functions are described below.
## Cues Outcomes Activation
## 1 BG_cat_white animal 0.03000000
## 2 BG_car_gray vehicle 0.03000000
## 3 BG_rabbit_white animal 0.04930300
## 4 BG_tree_yellow plant 0.03000000
## 5 BG_cat_brown animal 0.05861887
## 6 BG_car_blue vehicle 0.04911190Alternatively,the function getActivations can output all
possible outcomes per learning event.
## Cues Outcomes animal plant vehicle
## 1 BG_cat_white animal 0.03000000 0.000000 0.00000000
## 2 BG_car_gray vehicle 0.00970000 0.000000 0.03000000
## 3 BG_rabbit_white animal 0.04930300 0.000000 0.00970000
## 4 BG_tree_yellow plant 0.01910997 0.030000 0.00960300
## 5 BG_cat_brown animal 0.05861887 0.009700 0.00950697
## 6 BG_car_blue vehicle 0.02823568 0.009603 0.04911190We may want to add the activation of observed outcome in separate column:
act$Activation <- apply(act, 1, function(x){
out <- x['Outcomes']
return(as.numeric(x[out]))
})
head(act)## Cues Outcomes animal plant vehicle Activation
## 1 BG_cat_white animal 0.03000000 0.000000 0.00000000 0.03000000
## 2 BG_car_gray vehicle 0.00970000 0.000000 0.03000000 0.03000000
## 3 BG_rabbit_white animal 0.04930300 0.000000 0.00970000 0.04930300
## 4 BG_tree_yellow plant 0.01910997 0.030000 0.00960300 0.03000000
## 5 BG_cat_brown animal 0.05861887 0.009700 0.00950697 0.05861887
## 6 BG_car_blue vehicle 0.02823568 0.009603 0.04911190 0.04911190Note that getActivations only works for a single outcome
in each learning event. With multiple outcomes, please use one of the
other activation functions.
step 5: Visualization
Visualizing the change in weights between cues and outcomes is
facilitated by two functions: plotCueWeights and
plotOutcomeWeights. The first function retrieves the
weights between a specific cue and all outcomes (or a selection of
outcomes) for each learning event. The second function retrieves the
weights between a specific outcome and all cues (or a selection of cues)
for each learning.
oldpar <- par(mfrow=c(1,2), cex=1.1)
# plot left:
plotCueWeights(wm, cue="brown")
# plot right:
plotOutcomeWeights(wm, outcome="animal")
Both plot functions have a range of arguments that can be used to change the layout, as illustrated for the same two plots below:
oldpar <- par(mfrow=c(1,2), cex=1.1)
# plot left:
# 1. get outcome values:
out <- getValues(train$Outcomes, unique=TRUE)
out <- out[out != "animal"]
# 2. plot all outcomes, except 'plural':
lab <- plotCueWeights(wm, cue="brown", select.outcomes = out,
col=1, add.labels=FALSE, xlab='', ylim=range(getWM(wm)))
# 3. add plural:
lab2 <- plotCueWeights(wm, cue="brown", select.outcomes = "animal", col=2, lwd=2, adj=0, add=TRUE, font=2)
# 4. add legend:
legend_margin('bottom', ncol=4,
legend=c(lab2$labels, lab$labels),
col=c(lab2$col, lab$col), lty=c(lab2$lty, lab$lty),
lwd=c(lab2$lwd, lab$lwd), bty='n', cex=.85)
# plot right, different layout variant:
out <- getValues(dat$Cues, unique=TRUE)
out <- out[out != "animal"]
lab <- plotOutcomeWeights(wm, outcome="animal", select.cues = out,
col=alpha(1, f=.25), lty=1, pos=4, ylim=c(-.02,.2), font=2, ylim=range(getWM(wm)))
lab2 <- plotOutcomeWeights(wm, outcome="animal", select.cues = "brown", col='red', lwd=2, pos=4, add=TRUE, font=2)
Both plotfunctions are a wrapper around the functions
getWeightsByCue and getWeightsByOutcome. These
values could be used to extract the weights per cue or per outcome.
## animal plant vehicle
## 1 0.00000000 0e+00 0.000e+00
## 2 0.00000000 0e+00 0.000e+00
## 3 0.00000000 0e+00 0.000e+00
## 4 0.00000000 0e+00 0.000e+00
## 5 0.00970496 -1e-04 -9.801e-05
## 6 0.00970496 -1e-04 -9.801e-05Similarly, we can visualize the change in activations using the
function plotActivations, which is a wrapper around the
function getActivations.
oldpar <- par(mfrow=c(1,2), cex=1.1)
# an observed cueset:
plotActivations(wm, cueset="BG_cat_brown")
# an un-observed cueset:
plotActivations(wm, cueset="BG_cat_yellow")
Extra: Continue training
Another possibility worth mentioning is the possibility to continue training from an existing weight matrix.
## Shape Color Category Frequency2 Frequency1
## 1 cat brown animal 88 6
## 2 rabbit brown animal 88 1
## 3 flower brown plant 6 16
## 4 tree brown plant 6 56
## 5 car brown vehicle 1 88
## 6 bicycle brown vehicle 56 32We used the column Frequency2, and now we continue
training with column Frequency1. Note that in the new data
(rows 1 and 2, column Frequency1), there are much fewer
brown animals than in the earlier training data (column
Frequency2).
dat$Cues <- paste("BG", dat$Shape, dat$Color, sep="_")
dat$Outcomes <- paste(dat$Category)
dat$Frequency <- dat$Frequency1
# remove remaining columns to simplify this example:
dat <- dat[, c("Cues", "Outcomes", "Frequency")]
# add ID for learning events:
dat$ID <- 1:nrow(dat)
head(dat)## Cues Outcomes Frequency ID
## 1 BG_cat_brown animal 6 1
## 2 BG_rabbit_brown animal 1 2
## 3 BG_flower_brown plant 16 3
## 4 BG_tree_brown plant 56 4
## 5 BG_car_brown vehicle 88 5
## 6 BG_bicycle_brown vehicle 32 6After creating the training data (one event per row), we continue training. We will use the end state of the previous training as starting point for the new training. Two methods are illustrated in the code block below:
# continue learning from last weight matrix:
wm2 <- RWlearning(train2, wm=getWM(wm), progress = FALSE)
# number of learned event matches rows in dat2:
nrow(train2)## [1] 1194## [1] 1194# Alternatively, add the learning events to the existing output list wm1:
wm3 <- RWlearning(train2, wm=wm, progress = FALSE)
# number of learned event are now added to wm1:
length(wm3)## [1] 2388Now we can visualize how changing the input frequencies changes the connection weights. The red line in the plot visualizes the change in how predictable the color “brown” is for an animal.
out <- getValues(dat$Cues, unique=TRUE)
out <- out[out != "animal"]
lab <- plotOutcomeWeights(wm3, outcome="animal",
select.cues = out,
col=alpha(1, f=.25), lty=1, pos=4,
ylim=c(-.02,.2), font=2, ylim=range(getWM(wm3)),
xmark=TRUE, ymark=TRUE, las=1)
lab2 <- plotOutcomeWeights(wm3, outcome="animal",
select.cues = "brown", col='red',
lwd=2, pos=4, add=TRUE, font=2)
abline(v=length(wm), lty=3)
step 6: Activations
The activation of outcomes reflect the learner’s expectation that this outcome will occur, based on the present cues.
The edl package includes a series different functions to
calculate the activativations for outcomes:
activationsMatrix: Calculate activation based on a static state of the network. The basic activation function that is called by the other activation functions.
## animal plant vehicle
## 1 0.9907395 -0.009006675 0.02741258## animal
## 1 0.9907395# for a group of cuesets (all connection weights will be added):
activationsMatrix(mat,cues=c("BG_cat_brown", "BG_cat_blue"))## animal plant vehicle
## 1 1.769624 0.1189503 0.07837368activationsEvents: Calculate activations for each learning event in the training data.
# new dummy data:
dat <- data.frame(Cues = c("noise", "noise", "light"),
Outcomes = c("food", "other", "food_other"),
Frequency = c(5, 10, 15) )
dat$Cues <- paste("BG", dat$Cues, sep="_")
train <- createTrainingData(dat)
wm <- RWlearning(train, progress = FALSE)
# list with activations for observed outcomes:
act <- activationsEvents(wm, data=train)## Warning in activationsEvents(wm, data = train): Function will return list of
## activations, because in some events multiple outcomes occurred.## [[1]]
## food other
## 1 0.02 0.02
##
## [[2]]
## other
## 1 0.0298
##
## [[3]]
## food other
## 1 0.039502 0.049302
##
## [[4]]
## food
## 1 0.03920898
##
## [[5]]
## other
## 1 0.05793676
##
## [[6]]
## other
## 1 0.07677803# calculate max activation:
maxact <- lapply(act, function(x){ return(max(x, na.rm=TRUE)) })
unlist(maxact)## [1] 0.02000000 0.02980000 0.04930200 0.03920898 0.05793676 0.07677803
## [7] 0.09524246 0.09562928 0.11371669 0.13144236 0.14881351 0.16583724
## [13] 0.18252050 0.16495802 0.20722051 0.22307610 0.23861457 0.20496975
## [19] 0.12970522 0.25978388 0.15482098 0.27254565 0.22744455 0.29482029
## [25] 0.30892389 0.25685822 0.27172105 0.18378693 0.20011119 0.27574168## [1] 0.02000000 0.02980000 0.04930200 0.03920898 0.05793676 0.07677803activationsCueSet: Calculate activations for one or multiple cuesets.
# list with activations for observed outcomes:
act <- activationsCueSet(wm, cueset=c("BG_noise", "BG_light", "BG_somethingelse"))
names(act)## [1] "BG_noise" "BG_light" "BG_somethingelse"## food other
## 1 0.01000000 0.01000000
## 2 0.00980000 0.02980000
## 3 0.01960100 0.03950100
## 4 0.03920898 0.03871098
## 5 0.03842480 0.05793676
## 6 0.03765630 0.07677803## food other
## 1 0.01000000 0.01000000
## 2 0.00990000 0.01990000
## 3 0.01970100 0.02960100
## 4 0.02950499 0.02920599
## 5 0.02911290 0.03881888
## 6 0.02872865 0.04823951activationsOutcomes: Calculate activations per learning event for all outcomes in the data.
## food other
## 1 0.02000000 0.02000000
## 2 0.00980000 0.02980000
## 3 0.03950200 0.04930200
## 4 0.03920898 0.03871098
## 5 0.03842480 0.05793676
## 6 0.03765630 0.07677803