Collective Intelligence Design Challenges (CDL)

Via FotL I discover "Canonical Debate Lab (CDL)... researchers, developers, and system thinkers who have independently been working on collective intelligence systems for several years... goal is to aggregate contributions of diverse stakeholders into a unified information space, which can be accessed through multiple views... tailored to a specific use case... present differing perspectives and levels of detail. "

As they point out, there's already a lot of action in this space but not enough coordination and sharing. Some "challenges that lie ahead... often overlooked" include:

Cognitive Capacity: people don't have enough time and knowledge, so they can't be expected to read and learn everything before contributing their knowledge to a collective intelligence. Hence "information needs to be conveyed differently to people depending on expertise and background" (cf K4P Policylayer). But this can fragment information if it is not connected together so "users [with] necessary expertise or motivation can easily find and contribute information at the level of detail they are comfortable with".

Moreover, this is complicated by disagreements on the information itself: the more information there is, "the more diverse the set of perspectives ("multiple truths")" appear in the CI, particularly as each contributor's perspectives "carry ... unfamiliar concepts, cognitive biases, and competing terminology."

"Hierarchies of more abstract to specific representations [cf K4P Policylayer]... can be complex to reason about and often still need to be traversed to be fully understood" (not sure about this), "takes time to learn" (fair point) and "introduces additional sources of disagreement" (depends).

Map Diversity and Alignment: "multiple focused views of a shared information space are needed" to meet diverse needs, giving a sense of ownership to diverse communities, and reduce cognitive load. This however risks "fragmentation of the information space into multiple uncoordinated local views". My first reaction was that this would not be a problem if all views are based on the same data, but: "Self-organization is relatively straightforward early ... but when ... new people join the conversation, contributions multiply, sometimes altering the meaning of existing information" <- fair point, seen it happen.

Therefore "Local views should convey how much related information exists in other views, so users are invited to explore broader perspectives". And to avoid "the illusion of synergy through spurious connections... global alignment on key information represented across different local views is needed"

Context and Provenance: "For contributions to be understood as originally intended ... the provenance ... [of the info allows] semantic ambiguities and exact meaning of terms [to] be resolved." Even then, "conflicts will arise not only due to disagreement on explicitly stated information, but also due to differences in implicit assumptions ... about context which may not be shared".

Interpreting information thus requires interpreting the information linked to it, "and linked to that in turn, ad infinitum... If this makes it difficult for a single individual to interpret information in a CI system" it's worse for large communities "unless mechanisms ... negotiate ambiguity".

Quality and Trust: "Achieving high-quality information is an ongoing process" so the CI system needs to be "flexible enough for users to express any concerns they may have, and allow anyone to contribute freely". However this actually works against quality, requiring "mechanisms for ranking ... content", which must be "transparent, not overly restrictive, and ... be perceived as fair." (cf community triangle).

Concluding, "a proper solution is more likely to take the form of an ecosystem of interconnected tools rather than a single product designed by a single team ... [requiring] shared core data model and protocols".