Supporting awareness of content-related controversies in a Wiki-based learning environment

User generated content in Wikis is mainly distributed on the article view and its corresponding talk page. Potentials of analysing and supporting discussants' knowledge construction processes on the level of talk pages have still been rarely researched. The presented experimental study addresses this issue by providing external representations of content-related controversies which were led by contradictory evidence between discussants to foster awareness on socio-cognitive conflicts which can be beneficial for learning. Its aim is to investigate how increased salience of controversies can guide participants' (N = 81) navigation and learning processes. Three conditions differing in their degree of awareness support were implemented in this study. Results indicate that the implementation of awareness representations helped students to focus on meaningful discussion threads. Findings suggest that Wiki talk page users can benefit from additional structuring aids.


INTRODUCTION
Wikis such as Wikipedia provide a common and widespread opportunity to share user generated content that arose from collaborative writing processes on the Internet. In order to effectively collaborate together as authors and editors in Wikis and similar environments, some requirements that can Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for thirdparty components of this work must be honored. For all other uses, contact the Owner/Author. Copyright is held by the owner/author(s). OpenSym'14, Aug 27-29, 2014, Berlin, Germany ACM 978-1-4503-3016-9/14/08. http://dx.doi.org/10.1145/2641580.2641607 be derived from prevailing theories of writing and collaboration should be met. In reference to the Cognitive Process Theory of Writing [9], the composition of any text is structured hierarchically and does not follow an invariant order. According to this model, monitoring of processes such as planning, translating and reviewing text segments are required for successfully finishing one's individual goals.
Beyond individual premises for effectively organising one's own writing, collaborative writing systems have to provide basic technological requirements, optional individual roles (e.g. authors and editors) and activity spaces for monitoring processes on a group level [17]. Research in this area has shown that especially coordination in collaborative writing processes are crucial for the quality of resulting shared knowledge artefacts [8]. In order to facilitate cognitive processes of writing, Wikis could provide substantial functionalities that are required for the monitoring of writing.
Scardamalia & Bereiter defined knowledge building as the creation of knowledge as a social product [18]. A significant amount of research has been done on how learning and knowledge building processes can be backed by computersupported collaborative environments like online discussion forums, blogs or Wikis. Some researchers in this area were inspired by aspects of knowledge building processes in Wikis grounded on Piaget's constructivist school of thought [4]. According to the Co-Evolution Model of cognitive and social systems, analogous processes of internalisation and externalisation can be found on the individual as well as on a Wiki's system level and mutually influence each other. At every level there are manifold possibilities for conflicts to arise if either one cognitive system's knowledge base dissents the social system or vice versa.
In the last few decades, research has gathered various evidence that conflicts do not have to be detrimental for knowledge building and learning [7]. Socio-cognitive conflicts based on contradictory information of two or more cognitive or social systems can arise from content-related discussions led by evidence. Promoting to discussants the opportunities of perspective taking in meaningful talks, fostering of deeper elaboration processes and epistemic curiosity is possible. As a consequence, learning benefits on the individual and group level can eventuate by encouraging learners to constructively discuss controversies [15].
Wiki talk pages comprise hidden potentials for knowledge construction processes that can be made more salient by providing guidance to readers as learners in the underlying discussion threads. Potentially beneficial content-related controversies led by evidence can trigger socio-cognitive conflicts which were considered as meaningful, opposed to predominantly structural or socio-emotional conflicts that can also be found in such systems [11]. With the aim in mind utilising these discussed potentials to foster learning, two encouraging approaches of supporting such socio-cognitive conflicts on meaningful contents have been examined by recent research.
First, instructional designs through collaboration scripts have been shown itself as effective in terms of learning gains in different contexts [16] [12]. In recent Wiki research, explicit instruction sets to improve collaborative revision processes through scripting methods have produced promising results [19]. The successful implementation of a script collaboration in a Wiki setting has led groups to produce more coherent texts and to generate less redundant revisions. But collaboration scripts have to be designed cautiously [6]. The possible generation of unintentionally high cognitive load through over-scripting of individuals or groups might lead to unwanted adverse effects on outcomes.
The second promising approach to support potentially positive conflict outcomes originated in evidence-led controversies has been the deployment of cognitive group awareness tools. These tools which gather and visualise knowledgerelated information have been successfully implemented as implicit measures to structure collaborative learning processes [2]. Visual feedbacks as external representations of group awareness information have been realised as multidimensional graphs or highlighting of specific aspects of interest. Development and evaluation of cognitive group awareness tools assisting collaborative writing processes could be further supported by Natural Language Processing (NLP). Computational advancements and research on NLP In recent years yielded into more sophisticated libraries, tools and frameworks (e.g. DKPro TC) for analyses in Wikirelated contexts [5]. These developments allow researchers to conduct broader and deeper Wiki-based evaluations of text fragments as shared knowledge artefacts with special regards to the identification and processing of controversies and to possibly enable authors and editors to manage subpar structured information more effectively.
Supportive interventions for collaborative writing in Wikis have already been evaluated for the writer's task environment, i.e. setting a deadline or defining different goals of an individual's writing task, and led to a seto f extensions and design principles to facilitate group writing and constructivist learning processes [13]. Research on the implementation of cognitive group awareness tools to support controversial discussions has been conducted on online forums showed that making contributions more salient could strengthen a group's influence on others and foster learning processes [3]. Strengthening of a certain position (e.g. a minority group) could foster potentials to generate more innovative solutions to a problem by introducing more novelty aspects in contrast to the creation of possibly unwanted redundancies.
Due to the lack of recent research with regards to Wikis in this area, we were specifically interested in the mechanisms of action of a cognitive group awareness tool. Of particular interest has been, if the results that have been found by online discussion forum research could be transferred onto Wiki talk pages.Knowledge artefacts on discussion talk pages shall be made more salient to interested readers and different types of learners. As a result of the ver-satility of an individual's learning preferences, it was also expected that two cognitive key variables influence the extent on how controversies grounded on opposing evidences about the discussed content were handled. First, the Need for Cognitive Closure should determine how well an individual learner manages ambiguity in discussion [14]. Second, the manifestation of specific and diversive Epistemic Curiosity should be another component to predict information search patterns [1]. Both variables have to be examined as determinants to identify the best possible degree of individually needed support through a visualisation of conflict awareness information.
The resulting research questions can be briefly summarised as follows: • To what extent can meaningful evidence-led conflicts comprising contradictory user perspectives be utilised to foster learning processes?
• To what extent does the addition of explicit structuring measures to article talk pages guide learners and prevent redundancies of knowledge artefacts and facilitate the addition and integration of novel aspects?
• To what extent do one's personal need for cognitive closure and epistemic curiosity affect knowledge construction processes with regards to socio-cognitive conflicts on talk pages?

METHODS
An experimental study was conducted in a controlled laboratory setting with students (N = 81) at the University of Duisburg-Essen, Germany. A single independent factor variable with three levels was randomly varied across the study. The topic for this study's experimental Wiki was the mass extinction event of dinosaurs, because of the potential for opposing point of views and evidence for different theories. The three experimental conditions reflect the degrees of additional cognitive group awareness support on controversy information to the talk page and will be illustrated in the following: No support (C1).
The control C1 was inspired by default Wiki talk pages and did not provide any further information on a discussion thread, except for a title (Figure 1).

Plain conflicts highlighting (C2).
In condition C2 an external representation in the form of a single-coloured indicator was added. The indicators highlighted discussions with relevant evidence-led conflicting points of view on the article's topic (Figure 2).

Conflict status highlighting (C3).
Experimental condition C3 utilised two coloured indicators. Beyond the information of a discussion including evidenceled conflicts, the indicator informed participants whether a discussion contained an unsolved conflict (red) or solved conflict (green) (Figure 3). For this study, a basic Wiki article and 24 talks were generated. Six of these talks were designed as topics of interest comprising meaningful evidence-led discussion with contradictory arguments. Three of the interested talks contained a consensus at the end of the discussion, the other three talks ended in unsolved conflicts. The 18 residual threads contained either discussions on technicalities or off-topic talks. As dependent variables, data was collected on whole page reading times, individual talk reading times, talk clicking behaviour, article edits (quantitative and qualitative), discussion participation (quantitative and qualitative), measures of cognitive variables and learning success with a 15-item multiple choice test.

Conflict status highlighting (C3)
Participants were asked to read the basic article and to edit it at a later stage. Relevant information for completing this task could be found inside the discussion threads on the corresponding talk page with varying degree of supporting information on meaningful controversies between experimental conditions. At all major reading and writing stages, participants had to comply with compulsory time limits.

RESULTS AND DISCUSSION
Each participating student clicked and viewed on average M = 11.86 (SD = 5.27) discussion threads. Analysis of variance using planned comparisons with an orthogonal Helmert contrast revealed that in the control condition students clicked significantly more topics on the article's talk page compared to both supported conditions, F (2,78) = 3.80, p = .027, η 2 = .09. Detailed analysis on pooled reading times on the topics of interest showed differences in individual reading and selection behaviour (Figure 4). These results indicate that the implemented cognitive group awareness information leads readers in the intended direction and enabling them to primarily focus on relevant contents rather than residual discussion.
Participants in either of the supported conditions spent significantly more time on relevant evidence-led talks about the dinosaur mass extinction event. The students' preferences of first selecting the most relevant topics in both high- lighting conditions are further supported by sequential data mining using SPMF with CM-ClaSP algorithm [10] (Table  1). In contrast, control group participants showed the tendency to follow a top-down reading strategy, beginning with the very first thread on the talk page. This preliminary first look at most frequent closed sequential patterns underpins the previous result and conclusion that guidance towards the potentially most important discussion to fulfil the task worked as intended in both supported groups. Next, we investigated whether the degree of conflict awareness support led to a positive learning outcome, manifested in a higher multiple choice test score. Altogether, analysis of variance could not reveal any measurable difference between the three investigated groups, F (2,75) = .03, p = .968, η 2 < .01. The pure addition of conflict awareness support in the experimental conditions did not seem to affect the test performance at all. Preliminary analyses of the self-generated multiple choice test items indicate that several distractors were too easy and had unsatisfactory discriminatory power.
In consequence, some of the attractors had a relatively high probability of guessing the correct answer.
In more detailed analyses, we further investigated the test scores between the experimental conditions, considering the different categories of discussion threads as mediators in a parallel multiple single-step mediation analysis ( Figure 5). It can be shown in particular that those participants receiving varying degrees of conflict awareness information spent more time on reading unsolved conflicts, which resulted in significantly higher scores in the multiple choice test. Overall, at this stage of analysing the current study data, we can conclude that structuring Wiki talk pages by implementing cognitive group awareness representations related to socio-cognitive conflicts produced encouraging results. Participants who received any kind of structuring were more effectively focused on meaningful discussion contents led by evidence. Furthermore, under certain circumstances students were able to significantly benefit from guidance towards conflicts arisen from content-related controversies in terms of learning success. These findings should be taken into consideration for further developments and research on Wiki talk page discussions.
However, first analyses of the measured cognitive variables have been conducted to determine the best possible support for the individual learner and need to be analysed in detail. Likewise, content analyses of the produced knowledge artefacts are required to investigate if the additional support led to significant differences in text production quality.