What does it mean to have an ethical practice as practitioners of evaluation and co-design, in the context of innovation?

The origin of this section is the large number of questions that we received about formal ethics processes and the implications for the highly adaptive work of co-design and developmental evaluation as they relate to formal ethics. As we started to explore this topic, more questions and dimensions to the notion of ethics and the complexity of co-design and developmental evaluation emerged.

What became more clear for us is that ethics as a term is inherently cumbersome. Formal ethics in research is seen as a process, a committee, a set of standards, and an academic requirement, among others. It is off-putting in some contexts, seen to represent decisions about risk made by others in isolation from some stakeholders.

Essentially ethics are a code of morals about keeping people safe, thinking about the consequences of our decisions and practice, and conducting our work with respect and a sense of justice. For researchers ethical practice is about protecting the interests of those involved in the research, with specific attention to those who may be more at risk in the process, for example, children. It is also driven by the desire to publish, and by organisational and institutional risk management. Many professional associations also have ethical guidelines or a code of practice which sets out expected and appropriate behaviour for their members, for example, social workers in respect to the interests of their clients.

In understanding that all ethics frameworks are underpinned by a set of values and beliefs, this took us to focus on what it means to have an ethical practice as practitioners working in developmental, participatory, systems spaces, and what some of the gaps are that we need to think about to successfully navigate the complexity in this space. As well as keeping people safe, it is also about reciprocity, relationships, respect and creating space for bravery and courage. The assumption in this work is that we are working with community, families and frontline staff on something they care about, and in a way that helps them to achieve the things they are wanting to achieve.

This is an action agenda rather than a research agenda. And the process around how we do this is participatory and explorative. There is a purpose distinction between developing knowledge through research, the process, and the solutions and benefits intended in a co-design.

Exploring ethical practice through four lenses

In working in this space we have come to understand that action involves risk and at the same time inaction also involves risk. Consideration of ethical practice in this context extends to design, prototyping and implementation which involves change for families, communities and organisations.

It involves shining a light on current practice and working to shift into new patterns and behaviours between people and organisations. The complexity of this leads us to considerations of ethics in multiple and diverse ways.

With that in mind, we will explore these various dimensions and bring examples from our own practice through the following lenses:

The role of institutional structures and processes in supporting ethical practice
Practitioner readiness for ethics in innovation
Ethics and implications for prototyping
Ethics in the context of systems and capital

The role of institutional structures and processes in supporting ethical practice

This section explores some of the limitations of our conventional institutional ethics structures as well as how things are evolving.

Engagement with formal research ethics systems and processes in the context of evaluation is largely driven by funder requirements and industry standards and is common when operating within an academic or clinical setting.

In Aotearoa New Zealand and Australia a number of options have emerged to support and provide ethical oversight to research projects for those working outside of academic or clinical settings, such as the New Zealand Ethics Committee – Te Roopu Rapu I Te Tika. Some government agencies and more recently local governments have also started to operate their own Ethics Committees. These structures were predominantly intended to support applications for social research and evaluation, and often include a focus on data privacy.

More and more there is a need to extend such ethical oversight and support to developmental and design-led work which encounters a myriad of ethical considerations as part of involving people in research, design and change processes. This work is neither research nor quality improvement; developmental and design processes are often ongoing, have several stages and include aspects of implementation. They involve generating and testing things out and working with people to identify their definition and framing of issues and their priorities for change. This makes it harder to prescribe in advance what exact outcomes will be produced or what methods or questions might be applied as these may be developed in tandem with those participating.

The framing around and emphasis on ‘risk’ and ‘vulnerability’ often reflected by conventional ethics committees can also sit awkwardly with strengths-based developmental and participatory change practices. These processes often work at a faster pace than traditional academic research cycles and responses to service or system challenges may need to be addressed within days, weeks or months. While it is imperative that we keep safety and our duty of care at the center of practice, we also need to ensure our processes are reflective of what is needed to develop more family-centred and empowering culture. Bravery and courage are needed alongside a focus on safety.

We also want to avoid processes that inadvertently ‘design out’ the capacity for authentic engagement and better outcomes and capability building for families and communities in the process of risk management. We need the ability to think through a range of perspectives on this and balance legitimate concerns with other factors such as the capacity and agency that can manifest through participation. If the “co” of co-design is being honoured, the focus is on building relationships and relational practice, rather than positioning people as informants. Doing “with” rather than “to” people places a focus on capacity building and mutual benefit. This can raise different types of questions and challenges around, for example, confidentiality and how people might be involved over time than might be expected in traditional clinical trials for example.

While in some cases, teams may follow the conventional path of submitting an ethics application, other forms of support and oversight may be more suitable. Formal committees may be appropriate in some contexts, while others would do better with ethical practice guidelines and protocols. Such guidelines for design and developmental initiatives remain underdeveloped in the design and developmental evaluation space.

Ethical processes in practice

In response, formal ethics processes are evolving. Some are adapting to accommodate prototyping and emergent design. Some public service ethics committees are extending their functions in response to the growing needs and practice of design and translating how traditional ethics processes intersect, to support more collaborative, and ongoing design-led processes. This includes for example consulting and formalised support in advance of or outside the submission process. Providing advice early on and throughout the design initiative to help teams identify ethical issues in early framing and engagement as well as the prototyping and trialling process.

Some ethics committees are also extending their remit to include capability building, training and practice development of design practitioners to support ethical mindsets and awareness, and the appointment of Ethics Committee members with design expertise. In their efforts to explore what ‘fit for purpose’ ethical supports and structures might need to look like the Auckland Co-design Lab[1], in collaboration with other agencies and organisations, has identified the need for:

development of appropriate tools, processes and opportunities to support design teams to build awareness, skills and capability in principle and in practice – an “ethical heart and mind”
extending the remit of ethics committees to support in different ways including oversight and review processes that are responsive to the role and timing of teams working in this space
building awareness and shared understanding across design practitioners, those convening ethics and research committees and public service commissioning agents about design ethics and what support is needed and appropriate.

In a recent review of the concepts, materials and tools currently in use by several government agencies working in the design space in New Zealand, patterns emerged around the types of ethical tools and supports that were available. Mainstream ethics materials often included principles and guidelines, but the emphasis in practice tended to be on forms, templates and documentation derived from traditional research ethics. While there are values that underpin these materials they were generally implicit, with the exception of those developed specifically for or by Māori practitioners and communities.

Ethics materials developed from design contexts tended to be more visual, with a focus on tools and prompts that could be used for discussions within teams to explore ethical issues. One international example is for example theEthics Toolkit which endeavours to bridge the gap between philosophical and applied ethics for modern researchers, designers and developers’.

Consent forms in particular raise a range of issues. As initiatives become more complex in relation to data use, re-use and privacy, standard approaches adopted from research may not make sense in an evolving context. When participants consent to participating in a process, or to the use of their personal information, do they really know what they are consenting to? The conventional research single point of consent at the outset – like the consent forms designed for single research studies. This is problematic for participatory processes like design or developmental evaluation, because it is difficult to communicate to participants in advance what they are going to experience because we don’t always know. It’s often not clearly understood, by anyone, what participants are – practically – consenting to.

In reality, this emerges as the process unfolds. In light of this, consent forms may prove inappropriate to highly emergent processes. Instead, a more community development approach to consent may suit, whereby consent is discussed, negotiated and documented in an ongoing, transparent way.

In addition, because practitioners become immersed and intertwined in the everyday lives of the people they are working with, the boundaries between participants and practitioners and everyday life can become blurred in deeply participative processes – so the notion of consent also becomes blurred, particularly when there is an expectation that participants will be deeply involved in determining the direction of the process.

There can also be tensions between an action or change agenda and the kind of protection that consent is framed around, such as confidentiality and anonymity. Who gets to decide how to weigh up these things, especially if there is conflict or disagreement about either or both of these things? Consent processes, although they are ostensibly about respect and protection for participants, are also part of a process that often feels legalistic and designed to protect the researcher and the institutions with ‘real ethics’ being lost in procedures and bureaucracy.

There is room for different approaches or forms of community and professional ethics to oversee ethical issues in place. The principles of kaupapa Māori research are a leading light in this space for practitioners in Aotearoa New Zealand. Starting with values and principles from the outset, the issues they forefront for research also provide a framework for co-design and developmental evaluation. In conversations about ethics in co-design with Māori practitioners terms such as aroha (love), manaakitanga (kindness, hospitality, showing respect), reciprocity and tika (true, right, fair) are considered to guide practice. The integral nature of ethics and values in practice is demonstrated through this view of both research and design. For example a core principle of Māori research is to uplift the mana of Māori, and this is true also for design [2].

This discussion suggests a range of opportunities for how we might further develop our formal institutions and structures to better respond to the ethics of innovation practice.

Practitioner readiness for ethics in innovation

There is a lot already written about professional codes of conduct and many of the evaluation societies have formal codes of conduct for their members. These cover a wide range of expected issues, such as informed consent, individual autonomy, confidentiality, or how evaluators represent their skills.

In contrast, the social design community, in Australia and New Zealand at least, lack similar guidelines and development or alignment to particular ethical principles and guidelines is still nascent[3]. Mainstream design education hasn’t – for the most part – come with well-established ethics processes and there is not yet a professional body of designers to assist with standards or competencies [4]. There are some emerging efforts to respond to this, including the Ethics Toolkit mentioned above, and the RMIT design and ethics services. A number of communities and agencies are also developing their own guidelines[5].

Professional practice in this context requires high self-awareness of one’s own skills and capacities as an evaluator or designer, and a readiness to respond to possible emergent situations. For example, if you are engaged in interviewing, in some contexts, this process could be triggering and/or result in someone sharing something traumatic.

Professional ethical practice – especially in complexity – also requires that we pay attention to the nature of our relationships with all those we work with, including our responsibilities to each other. We need to find practical and tangible ways to value each person and their particular lives, hopes and aspirations. The most ethical we can be is in how we model good character and relational understanding in our own practice.

We can never assume our practice is neutral; we have to be extremely mindful of our motivations and assumptions, as well as the positions, values and perspectives of those who we are doing the work with. Inevitably, there will be multiple choices to make, all with consequences – good and bad – and the stakes will be different for different stakeholders in different situations.

For evaluators, the grounding for becoming a professional tends to focus on the technical and practical aspects, such as skills, competencies and the disciplinary knowledge to practice effectively. We tend not to emphasize the ethical and relational preparation we need to operate in complex, developmental or design spaces. How we demonstrate an ethics of responsibility to our fields – to prepare and support evaluation and design professionals with the realities of situated practice, amidst the uncertainties and dilemmas and the daily exercise of professional judgement requires that we pay attention to the following:

how to negotiate boundary decisions about evaluation design with multiple stakeholders, in ways that ensure equity and justice
how to ensure the relevant and important values are surfaced and embedded in our evaluation criteria
how to honour and access community wisdom and expertise
how to collaborate well, maintain trust, and balance power relations
how to productively manage conflict
how to bring our authentic, whole selves to the work we do in communities,
how to care for people in the process, and
how to be reflexive and sensitive to emergent ethical issues as projects unfold.

We cannot provide the answers to the ethical dilemmas of professionalism and professional practice with technique or lists of competencies alone. It is critical that we support professionals to cultivate their daily ethical professional judgement and build practice support to help professionals work through how they should act for the situated ‘good’ of those in each situation and context.

This means we need the capacity to think critically about a given situation—whether a situation is what it appears to be—and then to think practically about what should be done under the circumstances. To become experienced, we have to practice, learn and reflect on our experience, over and over if we are to develop the capacity to deal well with uncertainty.

Ethics and implications for prototyping

Innovation tells us to go fast and learn by failure. Rapid cycles of trying new things can enable new solutions, and can even be empowering to those involved. At the same time, rapid-cycle learning and the language of failing fast could also wind up being cover for doing things that might be harmful. There is a growing critique of prototyping such as that noted here by Cameron Norman on his systems changes and design blog:

A fair criticism of design is that it too often focuses on possibility without responsibility. Even on social issues we see design jams, hackathons, and ideation sessions that produce more ‘stuff’ (too often an ‘app’, as if the only solution to the worlds’ problems originate from a handheld electronic device) that is cool, sexy and disruptive without paying attention to what kind of disruption comes with that ‘solution’ [6] .

Prototyping in complex settings and with a focus on systems change tends to be just as much about changes in practice, complexities of relationships, revealing underlying behaviours and assumptions and grappling with perceptions and incentives in the system. Live prototyping and testing new responses in complex and sensitive settings introduce a wider set of ethical issues and considerations. Prototyping in this context, means working with families and other systems stakeholders to test things out in real-world contexts. Trying new things or testing changes in current processes to see what happens, experimenting with different approaches to getting the desired outcomes, identifying what does and does work on the ground and why, and evolving things in response.

Prototyping in this setting may include, for example, testing ways to get a poorly insulated houses upgraded, working with employees on prototyping pathways to transition into new jobs, or exploring new peer to peer capability models. One ethical consideration is being prepared to respond to what is revealed about the system, about people, about organisational practices and policy through this process. Unpacking interactions in a service system, first through research and discovery phases, and then through prototyping, often reveals things like duplication of effort, poor (or even illegal) practice by organisations, conflicts between the intent of policies and how they play out and interact in people’s lives. It reveals funding tensions between providers, distrust of and within organisations, and structural racism. It makes visible how services designed to support people are often actually making things worse and policies that are failing to deliver.

Ethical considerations in prototyping

This is true of research too and becomes even more significant when we are also trying to enable or influence change in that setting. Surfacing these things appropriately, working with these complexities and negotiating the potential dilemmas or conflicts that arise in ways that can ultimately be constructive and part of change, is part of the prototyping process. This requires careful and considered practice, including skills and resources on the side of the practitioners, as well as a willingness to get into and support working with the deep aspects of change on the side of the organisations and sponsors.

Another ethical consideration of prototyping in this context is the potential disruption to the lives of families and people participating. The new concept or outcome we are trying to learn how to create is unlikely to be immediately realised – yet the process of testing something may mean the status quo will be destabilised in the process. An example of this is in the Healthy Homes Co-design initiative in Auckland [7], a co-design process to work with families to develop interventions that would create warmer, drier homes. Rather than reduce risk by not trying things that had the potential for negative outcomes (such as loss of tenancy), they actively engaged with the complexity of the context taking considered action in partnership with families and other service providers.

Together, legitimate concerns of negative consequences were balanced with other factors such as the capacity and agency that can manifest through participation. When there was risk of negative outcomes, they worked with families to support them in whatever was needed to achieve the outcome the family was seeking. Managing this uncertainty and potential for negative outcomes required a high level of skills, networks and resources. Also critical was the flexibility to be able to respond as needed, and the shared commitment to do what it takes to achieve outcomes for families[8].

In a developmental space, the how is a critical aspect of the doing. The ethics of prototyping also involves the value that can be gained through the mutual learning and knowledge building process that occurs. As noted in earlier sections, many current mainstream ethics processes don’t assume there are benefits in the methodology (rather the benefits are largely anticipated to be from the research). Indigenous research and participatory action research differ in that such reciprocity is assumed. Formal mainstream ethics tends to weigh the benefits of the output or outcome against the possible risks of the methodology or process.

An ethical consideration of prototyping which involves trying, testing and building new knowledge together about what might work (and what success looks like), assumes the process itself should be a source of benefit. We have seen the potential for young people and families to build social capital, confidence and apply new wellbeing knowledge into their daily life as a result of being involved in design and prototyping processes. Through the design process, people can build their capacity for problem solving, knowledge translation, self-regulation and executive functioning as well as gain access to new opportunities. In a co-design, complexity space we must ask if there are ways that we can approach the work that optimizes the ethical considerations of the process and there is an inherent responsibility for capability building in prototyping. This may mean going slower or other costs.

Conventional perspectives on risk, and ‘avoiding harm’ can inadvertently act against the interests of families and close the space for trying new things. In complex and sensitive settings risk from taking action also needs to be weighed against the risks families face should their situation remain unchanged, for example living in unsafe housing. We need to rethink risk, as the status quo of many complex problems is not without risk. This is the space of both safety and bravery. If we are truly embracing the co in co-design, then we are talking about the possible risks of prototyping, we are building shared trust and understanding that lets us watch carefully what is unfolding and most importantly, make decisions together around if (and how) we might proceed.

Ethics in the context of systems and capital

The discussions above start to point to the kinds of structures, skills, resources and networks that are being drawn upon in participatory, community and systems work. It is well established that research in sensitive settings requires having in place the preparations and support needed to respond to what might surface, for example where people are triggered or need additional support as a result of engaging in the research process. Or where action needs to be taken in response to what is surfaced, for example if there is a risk of immediate harm. This is a form of social and human capital that researchers and organisations supporting such work know and build up over time, knowing what to do, who to call, how to ensure the right support is in place. Building on this we have been developing the idea of types of ‘capital’ needed for being prepared for innovation in complex and sensitive settings.

The kinds of networks, resources and courage that are critical to have in order to effectively mitigate ethical risks that might emerge in a prototyping process can, for example, be considered capital. If you are going to destabilize something, you need to be able to marshall resources creatively in response to situations that in some cases only become apparent in the process of change and experimentation. There is a need to expand the concept of prototyping and consider improving the overall safety net of resources and networks needed for working with families in complex settings when the stakes are high, and be ready with matching levels of skills and resources.

The resources and agility to respond, carry risk, manage, and at times ride out the implications of when things start to get unpacked, revealed or challenged in a system – even prior to prototyping can also be considered a form of capital. Design processes often reveal poor and bad behaviour, putting the spotlight on issues that may have been hidden or ignored. Teams are making in practice decisions about when to push hard into conversations or escalate things, and when to stay back, when to call people out and when not to. Practitioners and organisations need to feel they have the skills, resources and know-how and the backing of their organisations to work in these spaces.

In addition to needing the capital to operate in complex and sensitive settings, there are other ethical considerations raised when working in a ‘systems change’ setting. For example there is also the ethical considerations of starting something through a prototype that is not continued. The transfer of prototypes from within a collaborative, values and principles based co-design process, back into the ‘business as usual’ context also raises its own ethical issues. Prototyping happens through the creation of temporary spaces where we have permission and resources to try new things and work collaboratively.

This same environment, commitment, flexibility, and the underlying values and principles that enabled the prototype – don’t necessarily exist in the mainstream setting where we hope for things to be adopted. Efforts to scale often translate into taking parts of an intervention – not necessarily the values, essence or relational approach that might have been making the difference.

In seeking to shift the benefits or learning from prototypes into embedded and adopted responses and ways of working, we need to consider: what relationships and principles are integral to the initial success and what of these should continue to govern the future prototype/service development? And; is there readiness, structures, and capacity in the system for supporting new models?

Undertaking design and developmental work with families and communities that are experiencing cumulative stress and disparity will always require a powerful safety net to be in place. All engagements have the potential, or are likely to unearth experiences of trauma and strong emotional responses. Working with and being skilled and prepared for this is a starting point, and it is inevitable that other skills and resources – and capital – will need to be marshalled. In the complexity of a developmental process, it is impossible to predict exactly what this will be, thus, it is essential that we have strong networks with social capital, and contingency within our resources to be able to adapt and respond.

This includes organisational flexibility to commit resources to support emergent needs, and draw down on all the available resources and networks organisationally and at community level to respond as things unfold. Individual researchers or design teams don’t necessarily have this kind of capital. It needs to be built up over time. It requires networks and relationships in and across organisations and community, and individuals within organisations who are willing to be responsible in this way. To succeed, they must also have an appetite for risk and entrepreneurship within their own organisation’s structures and procedures.

In this work, we hold a bigger picture of learning new things and identifying new possibilities because we are seeking to intervene, understand and provoke responses and changes in the system. This creates another ethical tension that is the tension that comes from working at both immediate family and community level and aiming to influence bigger system shifts to achieve impact at greater scale through structural change. By their nature design-led approaches can help make visible the policy and system conditions that contribute to the challenges we are trying to address at service level. But traditional design (and other project management approaches) often identify bigger systems issues or levers as “out of scope”. This can’t be the case in social innovation work. Bigger systems issues can’t be out of scope, but at the same time we can’t only be looking for the bigger shifts and not addressing the immediate needs.

Destiny involves risk

We started this blog series at the beginning of 2019 with observations about the need to shift from assumptions and ways of working that are no longer working for us. We noted that if we’re serious about being able to create transformative change, we have to transform ourselves; transform our ways of thinking, acting and relating. The ethical lens is essential for guiding our stance as designers and evaluators, and the ethics of how we pursue change. It also means that our institutions and organisations must also evolve in step with changing notions of ethical practice in a developmental space.

Ethics in developmental and design processes is much more situated, contextual and embedded in everyday practice than it is about making rational decisions using abstract guidelines or principles, or following rules. it is about questioning and challenging, feeling and acting. Ethical practice is a process of constantly negotiating and working out what roles to take, questioning what we are doing and why, and building the necessary capital within a system to support those involved. It is a relational process, involving an ongoing ethic of care.

References

[1] See Discussion note Ethical Design Practice: Auckland Co-design Lab 2018.

[2] Pipi et al. A Research Ethic for Studying Māori and Iwi Provider Success(2004) Social Policy Journal, New Zealand, Issue 23, 141-153 See also Te Ara Tika.

[3] There is for traditional design fields like Communications, Interior Design, Landscape Design etc – but not so much for those working in the public and social sector, though some recent and emerging communities of practice are seeking to address this. Historically the focus on ethics has come in the form of “what” is designed i.e is the product ethical, not so much the process.

[4] A similar more international critique is also reflected here: https://censemaking.com/2016/07/05/ethics-and-systemic-change/ Ideo, for example, only recently published a book addressing design ethics in design research, https://lbodre.ideo.com/ long after most of their tools encouraging people into the field.

[5] See for example Ethical framework for Design from DMA and Ethical framework for Design from Paper Giant and Draft Ethical Standards for ICTD.

[6] https://censemaking.com/2016/07/05/ethics-and-systemic-change/

[7] https://www.health.govt.nz/publication/healthy-homes-initiative-auckland-codesign-stage2

[8] It’s not about fixing things for whānau, it’s about bringing the right resources around them so they can make an informed decision, and move towards their desired goal. Enablement.

This post was originally published on Dec. 17th, 2019 by the Centre for Community Child Health at The Royal Children’s Hospital (Melbourne) and the Murdoch Children's Research Institute.