Programming Tree task is one of the two task experiments planned for our project.
The second one being Matchstick task.
This experiment will examine the effect of transfer in a specific field of programming. Here, we will investigate whether manipulating the structure of information about the problem task induces creation of a certain mental representation and choosing of a particular strategy. We will also investigate whether the transfer of strategy used also occurs in neutrally introduced isomorphic task that does not induce creation of a particular mental representation.
We expect that structure of information plays a significant role in construction of mental representations and that some mental representations are more available (easier to think of) than others.
We have investigated the effect of mental representations a lot in our previous research, mainly how mental representations influence the problem-solving process. But now we would like to get some insight on their beginnings. A good idea is to begin investigation of mental representations from a very practical way is to consider brain as an open system (typical black box approach) and focus on the observable input and output. In this experiment we will focus on the input. Input in our case would be the given problem definition or task instruction (in the following text: instructions). There are a lot of ways instructions can be given - basically as many as there are known communication media. Although, we expect them to have some effect on the following problem-solving process, we do not expect it to be significant and therefore will not focus on it in the scope of our experiment (This can make a good investigation in the future). The way instructions are structured / formed is what interests us at this point. Kotovsky K. and Fallside D. (1988) have reported that the same problem - even with the same visual stimulus - can be influenced into different mental representations by slight changes in the instructions (see task). This change does not effect the problem definition in any way. With this experiment we aim to make a repetition study of their last experiment in a different setting as well as bring the world of computer science a little closer to psychology.
Subjects will be asked to solve a programming task in an object-oriented programming language regarding object relationships: parent-child and sibling. Their task will be to construct a tool (a program) that will be able to hold data and provide a framework to easily manipulate this data and calculate other relationships. The success of this tool will be measured by its performance in follow-up tasks and a subjective evaluation by another programmer. As said before, the tool will evolve around parent-child and sibling relationships, which will be implicitly formed in a simple tree structure - the restriction being that a child can have at most one parent. This task may seem a bit complicated to the reader, but it is one of the simplest tasks, as future programmers usually learn about it in the first year of university studies. This task was therefore chosen because of combination of its simplicity and ability to produce at least two different mental representations.
By studying this task we want to know whether manipulating the structure of information about the problem task induces creation of a certain mental representation and choosing of a particular strategy. We will also investigate whether the transfer of strategy used also occurs in neutrally introduced isomorphic task that does not induce creation of a particular mental representation.
For additional explanation on programmable trees, please refer this page.
We will investigate the first hypothesis with two different descriptions of the same task, and confirm it if the different information sheets significantly influence subjects to create the mental representation that is assigned to it. The second hypothesis will be confirmed if in the neutral task subjects tend to pick one mental representation significantly more often.
According to EU regulations, not confirmation from ethics committee is required.
A single signed document, that states that subject made an informed decision to participate in this experiment, suffices.
See the following study agreement
Study agreement - programming experiment.odt
When the subject signs the agreement, the experiment can begin.
All of the subjects will be tested with a single problem, described below. Three instruction sheets will be used in our experiment, each of them containing an isomorphic definition of the said task. Similarly, the subjects will be separated randomly into one of three groups and given the instruction set that is associated with their designated group.
| Group #1 | Group #2 | Group #3 |
|
Please read the following instructions carefully. Your tasks contains tree parts, that can be implemented in any order. First part of your task is to create a computer program in Java language that will be able to take a set of family relations data and sort it accordingly. The second part requires you to write a set of functions (specified below) that will be able to give correct answers when called. And the third part is to add the set of data to the program. The family relations are as follows: |
||
|
Benjamin is son from Alfred. Christopher is son from Alfred. David is son from Benjamin. Eric is son from Christopher. Frank is son from Christopher. George is son from Christopher. Hector is son from Christopher. Isaac is son from Christopher. Luke is son from Frank.
|
Benjamin is son from Alfred. Christopher is brother from Benjamin. David is son from Benjamin. Eric is son from Christopher. Frank is brother from Eric. George is brother from Frank. Hector is brother from George. Isaac is son from Christopher. Luke is brother from Isaac. |
Benjamin is son from Alfred. Christopher is son from Alfred. Christopher is brother from Benjamin. David is son from Benjamin. Eric is son from Christopher. Frank is brother from Eric. Frank is son from Christopher. George is brother from Frank. George is son from Christopher. Hector is brother from George. Hector is son from Christopher. Isaac is son from Christopher. Luke is brother from Isaac. Luke is son from Frank. |
|
Create a framework to contain this information and implement the following methods to return correct information: find all sons for person, find a grandfather of a person, find all grandsons of a person, find a random cousin of a person, list all uncles of a person.
The only restriction is that you are not allowed to use an imported libraries. This goes for Java utility classes as well. |
||
The subject is seated in front of a laptop computer with a running instance of an integrated development environment (IDE) Eclipse (the currently newest version is Eclipse Oxygen) or IntelliJ (the currently newest version is 2017.1.5), depending on their preference, and are provided with a couple of empty papers and a pen. Subject is also instructed to talk aloud and verbalize their thoughts as often and as in much detail as possible, The experiment begins when subject is given the information sheet and is timed until the subject says he/she has finished.
Experimenter sits next to the subject and makes notes about subject's thought process. If necessary, he/she asks a question to get subject to report more detail about his/hers thought process, without influencing the thought process itself.These probing questions should be limited and only used when subject is reporting insufficiently detailed thought process.
This task was selected because of its simplicity and frequency as a programming task.
Put simply, the task prevails in simplicity and familiarity.
Selection of an IDE is simply a personal taste and should not influence the produced code. We chose to use an IDE for faster performance on the test, as subjects can run and test their code at any time with a click of a button. Using an IDE instead of a simple editor should simplify the task without influencing it.
In addition to those attributes it also has 2 commonly identified distinct mental representations, which we believe can provide significant difference in the problem-solving process. But we will see that after the experiment is analyzed.
Task as such can also serve as a good repetition study of Simon's study (Kotovsky & Fallside, 1998) and thus further solidify or break their findings.
The in depth process of building this task can be found on this page: Simple programming structures
The data that will be collected:
They will be analyzed to and compared to other transcripts of (a) the same group and (b) other groups. This will help us identify if subjects' thoughts were aligned with the mental representation that is associated with the produced code. Additionally, we will be able to see if subjects are creating a previously unknown representation.
Additional measurement.
Program will be analyzed for its efficiency through code review and performance statistics.
| Group #1 | Group #2 | Group #3 |
|
Represented above is the (binary) array tree node. |
Represented above is the list tree node. |
Both representations from groups #1 and #2 should have equal probability to be created and should be equally good at solving the task. There is no preference towards any of those two mental representations. |
For better representations, here are visual representations of programmable trees that can be built using the above mentioned nodes.
| Array tree node | List tree node |
 |
 |
Representation and Transfer in Problem Solving.pdf
TODO: Literature on program evaluation has to be added.
Is this experiment to rigged? We are trying to create a task that will not be biased towards any of the two mental representations. But at a first glance it seems somehow too rigged.
The neutral instructions (group #3) is just comprised of both #1 and #2. Is it really neutral? Should be, but is it really comparable with other groups?
