Relational Dynamics, Values and Worldview Clashes – Exploring Systems Practice – TB872 Part 2 Week 8-9

(This post is longer than others I have posted, as I was not able to complete my work and post during the week of Christmas. This post contains my notes for both Week 8 and 9 of the ‘TB872 Managing Change with Systems Thinking in Practice (STiP)’ module at The Open University, including review of the book ‘Systems Practice’ by Ray Ison.)

Happy new year to you all! I hope you all have had a wonderful festive season building up to the new year. A lot has changed in my life in the last year, and I am still adapting with the hope that whatever I learn from challenging situations now will make things easier down the road—a real life recursive learning process! The knowledge that improvement is almost guaranteed (unless I really wasn’t paying attention), helps to keep some of the heavier depressive symptoms and anxiety-induced meltdowns at bay. It only became knowledge to me after I had pushed through the most challenging situations, and sometimes that is the most difficult bit especially when evidence has shown that things often get way worse before they get better. Every situation expanded my toolkit for fixing the next thing that might come up, and the confidence of knowing that my arsenal of skills, tools, and ideas for how to manage different situations is more powerful now than it has ever been. If you have had a rough year, I hope it also brings you comfort to know that with each passing day, even without trying, you are stronger and more prepared to handle whatever may come. With awareness of our own strengths and weaknesses, skills and knowledge, we can expedite our growth and overcome our hurdles.

I am aware of my limitations in regards to keeping up with my studies, for example it takes longer for me to process information than it used to and I quite quickly forget a lot of it without frequent review of the information in a variety of formats. Although my assessment with StudyTech for disability support was completed last month, it seems Student Finance England has still not approved the recommended support (two months after starting my current module). I am making do with my usual strategies including routines for managing my time and energy levels. I try to complete work within recommended timeframes, if not earlier, and I am glad that I read the required books early on. We are now having to refer to the books in detail and although I don’t remember a lot of what I read, I did get a good gist of what to expect on the course and I left a bunch of useful notes in the margins for my future self!

‘Working out’ with Systems

Ray Ison is one of the designers of the Systems Thinking in Practice (STiP) courses at The Open University (OU), and author of module materials such as the online course content and related books such as Systems Practice: How to Act (in situations of uncertainty and complexity in a climate-change world. Chapter 2 of Systems Practice (Ison, 2017) begins by reinforcing the view that systemic thinking can only be developed through practice, experience and mistakes (Sterlein, 2004). We are invited to use the book to create circumstances where it is ‘safe to fail’ while developing our systems thinking and practice, in order to maximise learning. As I have stated previously, I believe it is a privilege to be in situations in which we have power to manage change let alone have minimal consequences from any failure. A huge part of the module involves the setting up of circumstances to practice STiP. Although it is important to be able to identify situations suited to STiP, and useful for creating opportunities to practice STiP (if it is even possible), I still argue that Systems Thinking should be taught as an apprenticeship or vocational qualification rather than presenting as a theoretical / philosophical degree that happens to be useless without practical application.

There are some strange statements in the book that lack scientific backing, and should have some proof if gleaned through years of systems practice. For example, Ison claims that systemic sensibility (appreciating interdependencies over cause and effect relationships) is innate in many people but is lost through conditioning. The author laments that their job of helping people recover their systemic sensibility would be easier if non-systemically-thinking people were not always asking for ‘explanations… justifications and evidence that ‘it works’. I think there should be evidence that the premise and purpose of a curriculum is valid, and that the proposed solution for the world’s most complex problems can be proven to be beneficial to practitioners and stakeholders. This does not mean that I do not see the value in STiP, or systems thinking in general, but there are some questions that must be asked about the development of the curriculum and how it can be improved if the goal is to encourage more people to learn about systems thinking in future.

The idea of abandoning certainty and being open to our circumstances is useful and potentially profound advice, aside from being a suggestion for how to create circumstances for experimenting with thinking and acting systemically. I intuitively understand that Ison is right about the possible generative and innovative outcomes from being open-minded. I can also see how, despite Ison claiming to have no intention of defining STiP, it would be difficult not to definitively define the subject for the purpose of teaching one’s ideas about it. For example, in Systems Practice (p. 23), we are provided a list of systems concepts that are ‘likely to be experienced when encountering a systems practitioner’. From this we can assume that the systems practitioner being referred to is of a certain school of thought from the United States / United Kingdom / English-language Systems Thinking traditions. There is also a focus on, and preference for, certain Systems thinkers and lineages (such as those that converged at the Macy conferences on systems thinking in the 1940s and 1950s), which further narrows the boundaries of what Systems Thinking is within the context of the Systems Practice book, and other module materials for Systems courses at The Open University. On page 45, The Open University’s Systems course definition of a System is outlined as:

• A collection of entities
• That are seen by someone
• As interacting together
• To do something.

A reading from an article by Simon Caulkin (2007) is provided as evidence of what a systems thinker sounds like—he uses the right ‘Systems’ terms and references a scholar within the right sort of historical lineage to make sense of contemporary circumstances. According to Ison (p. 36), ‘by his use of the concepts he shows, to me at least, that he understands what he is talking about i.e.., he displays both systemic sensibility and systems literacy.’ Ison does acknowledge the importance of social conventions and institutions in developing our individual intellectual lineages or ‘traditions of understanding’, so I will continue to explore the course materials further with the understanding that it was designed from the perspectives and ‘traditions of understanding’ of specific individuals but do not constitute the extent of Systems Thinking as an intellectual field. With this in mind, Caulkin’s article is useful for highlighting systemic issues (and solutions) that played a part in various situations.

I find the discussion about situations and systems in chapter 3 very interesting. Based on the OU definition of a System in Systems Practice (Ison, 2017, p. 45), people in the same situation may experience different elements and interactions of a system. Due to their differences in traditions of understanding and the different boundaries they set to determine a system of interest, each system model will be different for every person. With awareness of being part of all situations we engage with, we can decide to make conscious choices that define a system. Therefore, a system is conceptual—not a reflection of the situation as it is, but of what we decide is useful for the purpose / goal of the system. ‘…‘Bringing forth’ a system in a situation is a particular way of knowing the situation.’ So far, this is the most valuable information from the TB872 module and the course material. It aligns with my own principles about developing awareness of our role in systems in order to make purposeful choices that benefit the system, but, importantly, it reminds us to try to develop this ongoing awareness as a practice. We are given a framework for systemic inquiry that can help us make the most of our developing systems thinking and practice, which I think could be useful (with practice!)

Social Relations and Relational Dynamics

Ison (2017) suggests that what constitutes systems practice arises in social relations. From my understanding, this is because our traditions of understanding are constructed with social contexts, and the choices we make in our systems practice is partially determined by social dynamics. There are not many situations in which we as humans exist and engage with, that are not either affected by or affect other humans in some way. For example, if I reflect on my own practice I can see that:

• Most of what I have learnt in my life is directly or indirectly from other people
• I exist in a society that has certain social norms and expectations
• I am dependent on social systems for survival and wellbeing
• I have been greatly influenced in my worldview by social and educational institutions
• Many of my decisions regarding my actions or continued practice are made with other people in mind because most of the situations I engage with are interconnected with the lives of others
• Social feedback guides my understanding of what is acceptable and desirable behaviour, and what might be systemically beneficial
• Even in situations where I am in defiance of social norms and expectations in my practice, my decision to act in defiance is still based around social dynamics.

I wonder what kind of situations might exist outside from social dynamics… if social impact can be minimised with awareness of our influences, and if having less empathy makes for more ‘logical’ thinking or just results in practice that lacks consideration of an important system element.

If we refer to the systems lineages heuristic in Systems Practice (Fig. 2.3, p. 32), we can see that many of the contemporary cyber-systemic approaches are influenced by a variety of intellectual fields. Modern understanding of any subject, including Systems, is influenced by other thinkers. The different lineages and approaches continue to influence each other, and continue to be applied in a wide range of contexts by people from many different backgrounds. This can be seen to support Ison’s statement that systems practice arises in social relations: Learning is effective when continually built upon—when we purposefully think and act systemically, our cultures, societal influences, and access to information from others become stepping stones for development.

The lineages diagram lists systemic approaches on a spectrum from ‘systems as ontologies’ to ‘systems as epistemologies’. Ontology can be seen as the ‘study of being’, concerned with what actually exists in the world about which humans can acquire knowledge; epistemology is the ‘study of knowledge’, concerned with all aspects of the validity, scope and methods of acquiring knowledge (Moon & Blackman, 2017). Approaches labelled as having ontological views of systems include Systems Analysis, Systems Biology, Earth Systems Sciences, and Systems Engineering. These approaches use systems models to understand physical subjects and their interconnections, whereas approaches that view systems as epistemologies use models to understand and develop different ways of thinking. Epistemological systems approaches include Critical Systems Heuristics, Soft Systems Methodology, Operations Research and Management Sciences. Each approach has its traditions from its lineages and origins that have changed over time, adapting with new ways of thinking, new technologies, new social norms, and the new needs of social groups and the people that form them.

Valuing Systems Practice

The final chapter of Systems Practice: How to Act (in situations of uncertainty and complexity in a climate-change world) by Ray Ison summarises the book, focusing on why and how we can/should choose to ‘do Systems’. As the title suggests, Ison refers to climate change as a factor contributing to uncertainty and complexity in our shared world, and a reason for creating conditions for systemic thinking and action. Ison believes that investment in systems practice, especially within supportive systemic contexts, offers significant opportunities including:

• qualitatively different practice
• greater variety of options for managing complex, uncertain situations
• strategic strength in a co-evolutionary world.

A recurring subject is that of hope but also the necessity of fostering practice that focuses on how we do what we do now rather than just imagining a future. We can choose to be open to growth and transformation or we can choose to give in to fear, becoming defensive (denying confusion) or destructive (giving up in despair). Embracing complexity, uncertainty, learning and inquiry opens up more choices to us as practitioners. Doing so requires re-evaluation and reframing of our views to see complexity as an opportunity (see Fairtlough, 2007).

Our aesthetic judgements reveal something about ourselves, and the similarities between us are the reason for the existence of archetypes. Awareness of the patterns makes it possible to act with wisdom (see Bateson & Bateson, 1987). Patterns/relationships reveal more than descriptions or supposed ‘facts’. With this logic, systems in the book are seen as tools for analysing relationships, rather than descriptors of the world.

For Ison, committing to action in the world by doing Systems (rather than just being systemic) is important for understanding situations, making judgements, and having productive conversations. The transition from being to doing involves making conscious choices, taking purposeful action, and learning how to further improve situations. Systems practice, then, is learned and enacted through doing.

Using the ‘juggler’ isophor, the transition from ‘being’ systemic to ‘doing’ Systems encompasses all the ‘balls’ juggled by an aware systems practitioner—Being, Engaging, Contextualising, Managing. The ‘juggler’ is an isophor for embodied systems practice (doing Systems), presenting an ‘ideal type’ of systems practitioner that can be used to learn about our own or others’ systems practice. Evaluation of our own practice is emphasised, since huge change resulting from paradigm shifts can happen when people begin thinking about their thinking.

Evaluation as systems practice is contextualised within a situation and performance in the chosen system of interest. Performance may relate to effectiveness (why), efficiency (how), and efficacy (what) of purposeful, relational behaviour within the context. As described by Armson (2007), a system of influences create the emergent property of a person’s perceived performance in systemic practice, so contextual models are useful for evaluation of a situation including:

• the practitioner’s history
• their chosen framing of situation
• the practitioner’s approaches
• the stakeholders.

The evaluative process would be enacted through praxis, with reference to systemic models of causation. It is best to consider multiple perspectives of stakeholders, valuing difference in understandings, and enabling emergence of new/different narratives / emotional dynamics. Ison (2017) believes that evaluation is something that needs to be practiced as an ordinary everyday part of what we do, not necessarily left to experts. However, there are alternative definitions mentioned that are commonly applied in practice and may be relevant to specific situations that are not particularly complex or not requiring continuous learning. For example, evaluation may only be used in some settings to guide decision-makers (OECD, 1986), or it may be more like an audit type of review (Wadsworth, 1991)—systematic, objective and standards-based rather than systemic, illuminative, process- and learning-based.

Evaluation can happen without awareness of the process, for example, noticing discrepancy between expectation and reality. With awareness, systemic evaluation enables us to ask powerful questions, explore inter-relationships (how systems behave), perspectives (motivations and how people behave) and boundaries (what is valued, and what might be affected by interventions), (see Williams, 2009, and Reynolds et al, 2016a).

Value is said to not exist in itself, but is emergent and subjective—it arises through praxis, grounded in our activating emotions, arising from unique traditions of understanding. Therefore, evaluation not only brings up questions of value, but can extract value from a process. Typically, culture, context and language define value, for example in framing ecosystems by economic value in context of a capitalist market versus framing ecosystems’ ecological value in the context of global biodiversity, our understandings and actions may be considerably different. Our priorities for action arise from our developed values. Values are conserved through tradition—different traditions of systems thinking have different practices because of different perspectives on the importance of maintaining versus dismantling traditions. Some critical thinking approaches to interdisciplinarity in systems practice value epistemological pluralism, recognising different ways of knowing that can all be valued and accommodated. These approaches see systems as interwoven rather than dualistic.

We are encouraged to build systems practice capability including personal mastery of sense-making, practice, theory, and praxis (relating to the four ‘juggler’ balls—Being, Engaging, Contextualising, Managing) and taking action within our domains to create suitable conditions for systemic thinking and practice, and ultimately systemic change. An heuristic for designing learning systems to build systems practice capability is adapted from Jones et al (2009). The learning model has different considerations and outcomes for domain-specific and generic modes of practice (see p334). Although the model can be seen as reification of the valuing of systems practice capabilities (as agreed on by some experts in the field of Systems Thinking), it is promising to see an acknowledgement of different ways that Systems could be taught depending on the purpose or setting.

Ison identifies some of the constraints practitioners may face, but ignores others. He recognises that enhanced performance cannot always be achieved due to limitations within structure, processes, relationships etc. and explores the ‘responsibility ←→ response-ability’ relational dynamic, where institutions may constrain change by not providing circumstances for response-ability. Ison suggests that an effective systems practitioner can take a ‘design turn’ to reconfigure the relational dynamics of the situation.

I think that most people that encounter such constraints cannot simply take design turns, or change the structures and cultures within their situations. The power imbalances in hierarchical and capitalistic structures are often not well suited to enacting change that is not beneficial to the most powerful in the hierarchy rather than the system as a whole. This is an ongoing concern that I will likely keep bringing up as I feel that it is not addressed in my course materials. Ison does briefly mention it but states ‘… situations such as climate change are best understood as non-equilibrial yet for decision making we rely on a paradigm that is built on concepts of equilibrium. This is significant as the treasuries of the world are, in the main, places where this type of thinking is institutionalised. However, the failings of economics and the mainstream ways of valuing are not my concern here.’ If we are really contextualising systems practice ‘in a climate-change world’ as the title of the book suggests, perhaps the failings of economics and mainstream ways of valuing should be a concern.

Ison argues that we live in language, defining our shared realities through our unique lenses. Through authentic conversation we can create the circumstances for building relational capital, ‘that form of capital that synthesises the other forms of capital (natural, manufactured, social, cultural and institutional)’ (see SLIM, 2004). How often are most people in the position to enter into, and develop authentic learning relationships particularly in situations that affect multiple stakeholders and require systemic change? Perhaps it is just my experience and resulting cynicism, but it is a massive privilege to have your individual beliefs, values, and experiences respected in situations where systems thinking would be useful such as in organisational settings. The suggestion to ‘experiment with breaking out of hierarchy in organisations through building capability for systems practice’ feels out of touch to me. Where are the practical tips for how the average person should begin to act systemically aside from the work they can do on themselves? Is this information just not for those people? Have I just not got to that part of the course yet? I am very curious to find out how ‘…reflexive systems praxis has the potential to reconfigure relational dynamics in situations of concern and is thus a praxis that can ‘undo’ configurations, dispositions and discourses of power.’

The book ends with a list of ‘opportunities’ for ‘building systems practice within a discourse of hope in a climate-changing world’, inviting the reader to consider the points in light of their own circumstances and thus taking a ‘design turn’. There are four out of seventeen points I personally could feasibly action at this time, and I am sure for most people it is even less:

• If your current systems practice is a ‘silent practice’ then take actions to ‘come out’; acknowledge good systems practice when you experience it
• Create opportunities to do more systemic action research and at the same time improve quality and conceptual rigour of your practice
• Appreciate relational thinking and act with awareness that humans are engaged in a co-evolutionary dynamic with the earth. Take responsibility for the futures we can ‘design’ (i.e., embrace a design turn)—but can I take responsibility for this, even if I want to?
• Seek to recover systemic sensibility wherever it lurks, build systems literacy and invest in systems thinking in practice capability – what is meant by ‘invest’ in this context?
• Contextualise and add to this list as you see fit.

Are we relying on systemic change to occur only by those that have the power and resources to enact it? Do those that have power, and benefit from the status quo, have incentives to inquire of all affected stakeholders? How do those that are most negatively affected by systemic issues make systemically beneficial change?

Emerging Worldview Commonalities and Clashes

I have noticed that there is a potential worldview difference amongst people involved with the TB872 module, although it is possible that there is just a misunderstanding between us that can be addressed through discussion and reframing. It would be interesting to conduct qualitative interviews to ask about different traditions of understanding, and theories of change. I think this would give great insight into how different people imagine change could or should happen, what they consider to be possible and realistic.

I am acutely aware of my lack of power to enact change in complex situations, because of my current situation—being disabled, not in employment with others, not in control of much that might affect other people’s lives in any significant way. For the tutors and many other students on the course, it is a given that they go to work and are in charge of managing something or have the power to advise others on how to make changes (and be taken seriously by those that can make those changes). To me, this feels like a strange assumption. Maybe because of the type of person that would sign up to take a post-graduate degree in Systems Thinking in Practice, it is assumed that they must be working in a professional setting that allows them to test out Systems Thinking practices. This is quite a logical assumption, but is an unfair generalisation if those that are not in such a position are not equipped with tools to utilise the resources available to them within their individual contexts.

In a recent tutorial on Systemic Inquiry into Systems Thinking in Practice, I wrote the following in the chat box and had responses from other students as well as the tutor (some unrelated messages removed):

Me: sounds like it would be important for stakeholders to have similar openness to possibilities of change, and not have their own agendas. Or necessary for us to have power to make these inquiries and design turns 

Tutor: Stakeholders are always important, Alex, so yes. 

Me: doesn’t feel realistic though, at least in my experience 

Tutor: Yet, Alex 😉 

Me: 😀  

Student: I agree Alex, it’s hard to implement in practice – being open and transparent about it leads to people feeling uncertain in my experience.  
Student: Do we have a little time to quickly discuss HOW to do this in practice? 

Me: yes, it is great for evaluating our own practice, but harder in wider contexts 

Student: With others I mean.  
Student: You are generally brought into situations though where they are driving for a particular solution.  

Me: thanks, I guess I am thinking about it within limitations of situations I have experienced 
Me: not everyone will have those limitations 

Student: Me too Alex. Just from experience so far 
Student: Awaiting for the STiP movement to be more acceptable! 😀  

Tutor: That is when you have to be little more creative with design turns in your practice, [student name]. There is always wiggle room. 

Often in other messages with tutors and students, such as on the module forum, referring to work situations is frequently the first thing recommended. I understand why, since a workplace is where most people might use systems thinking to tackle complex issues with many stakeholders. I also understand that systems thinking can be applied to almost any situation we are involved in, and that has been emphasised by tutors. However, it feels to me like there are a huge amount of possible variations when it comes to ‘situations of concern’, but so far not much guidance on how to apply systems thinking in such a wide range of situations.

Is STiP for any possible situation, or is it best used for managing change in messy situations with multiple stakeholders? Although both may be true, it seems we will be assessed on our STiP capability with the latter, so it doesn’t make sense for us to talk about the wide range of other possibilities in the context of academic assessment (it is extremely valuable for personal development in any context though!). Those who are not in the position to demonstrate their capabilities in suitable situations will have to do so hypothetically, with awareness of the limitations of their situation, history and understandings, according to the module material’s description of ideal practice using STiP. This espoused expectation of how it is best to practice does not address systemic inequalities, it only somewhat accommodates differences. This suggests potential of there being blind spots in regards to issues of class, and accessibility of marginalised or less privileged groups to power and opportunities to make change.

The introduction of a STiP apprenticeship scheme in the UK bridges that gap in many ways, providing the opportunity for an apprentice/student to have exposure to suitable situations that they may manage with STiP while they develop their understanding through learning materials. This accurately reflects the ‘braided learning’ technique that is employed to support students in a circular, recursive learning process where the development of STiP, and the practice of managing change with STiP in a situation of concern, feed into each other. An apprenticeship also provides real world experience and professional networks that can open doors to future opportunities. Unfortunately, access to the apprenticeship scheme is limited and there are few opportunities of this kind at the time of writing.

I was surprised to have experienced what felt like a dismissal of my concerns regarding my perceived lack of power to enact STiP, which just seems to grow as I have progressed with the course. I recognise that I am lacking experience and knowledge of what the rest of the module will reveal, so perhaps our tutors know something that I have yet to realise about our power to enact change. I am looking forward nervously to the focus in Part 3 of TB872 on social learning, as I imagine there will be greater opportunities revealed for change made collectively. For those that lack societal sway, realising collective power as the primary mode for enacting change could be a powerful lesson.

A Meta-Narrative for Systemic Change

In my previous blog post, I discussed the systemic inquiries that we are required to undertake as part of the TB872 module. To briefly summarise, Systemic Inquiry 1 (S1) involves developing my STiP capabilities in conjunction with Systemic Inquiry 2 (S2) which requires us to manage change in a situation of concern.

With feedback from my tutor and other students on the module forums, I narrowed down my options for Systemic Inquiry 2 to:

1. Managing Diversity and Inclusion Challenges in Postgraduate (Systems Thinking) Curriculums
2. Managing Online Collaboration and Group Work Challenges for Disabled Students
3. Managing local community development systemically via non-profits / governmental organisations

Option three would probably require a lot more work to access information regarding my local community and active organisations in my area, which would be really interesting if I had more time and mental capacity. Out of the first two options, exploring diversity and inclusion challenges would be the most interesting to me, and in line with my personal values arising from my traditions of understanding. I am not yet sure if I should focus on Systems curriculums only, or widen the scope to postgraduate curriculums in general. I suppose I could limit my investigation to the curriculum accessible to me at the moment for the purpose of developing my STiP capability within the context of this module, but later suggest how I may generalise my inquiry within wider boundaries.

In my opinion, based on my experiences and understanding of education systems, it is beneficial to design curriculums that draw from a wide range of current understandings of a subject. An out-dated or narrow view is limiting for students and is detrimental for development of the academic/professional/intellectual field. Institutionalising limited knowledge has profound systemic effects for societies, whether intentional or out of ignorance of the benefits of varied perspectives. Systems Thinking, as we have learnt through the TB872 module, advocates for the consideration of various perspectives to understand the values, thinking styles and processes, and actions of ourselves and others. When we understand why we do what we do the way that we do it, we can make informed purposeful decisions, and manage systemic change more effectively.

There may be many reasons for not including varied perspectives in educational materials, but doing so should be questioned if we value epistemological plurality and outcomes that provide systemic benefits. Our education informs our practice, and a lack of frameworks is a lack of tools at our disposal. Furthermore, intentional exclusion of certain viewpoints in curricula sends a strong message about what the experts and thought-leaders in those areas consider to be useless or at the very least, less important for students to learn. This is how status quo is maintained in academia, and by extension, in professional practice and institutional structures that continue to marginalise certain voices while uplifting others.

Another way that exclusion in academia may be enacted is through lack of accessibility to certain groups of students or experts. For example, courses may not be financially accessible, or do not cater to the needs of differently abled people, or there may be ideological barriers such as racism, homophobia, or political differences. This kind of exclusion may also be intentional or resulting from carelessness, but is harmful in either case.

The function of my inquiry would be to identify ways in which the purpose of education is more effectively fulfilled with consideration of the needs of marginalised individuals in the education system, and the ramifications of limited/exclusionary curricula on marginalised groups within and outside of educational settings. To evaluate the effectiveness of any changes to the situation of concern, I would in this case have to define what different stakeholders consider to be an effective curriculum. It would also be important to know whether they consider there to be significant impacts to themselves relating to equity, diversity and inclusion within their educational settings and curricula, and how this may affect their continuing academic, professional and personal lives.

Evaluations of expected educational experiences versus real lived experiences of students would be needed in order to measure effects of any changes. In my personal ideal situation, I would hope for transformation towards a more inclusive, diverse, and equitable educational system within the context of institutionalised learning about how to continuously develop curricula. A curriculum should, in my opinion, provide the most possibilities for opportunity, for the greatest possible number of individuals.

References

Bateson, G. and Bateson, M.C. (1988) Angels Fear: Towards an Epistemology of the Sacred. Bantam Dell Publishing Group.

Fairtlough, G. (2007) Three Ways of Getting Things Done. Triarchy Press.

Ison, R. (2017) Systems Practice: How to Act (2017). London: Springer London. Available at: https://doi.org/10.1007/978-1-4471-7351-9.

Ison, R., Blackmore, C. and Armson, R. (2007) ‘Learning Participation as Systems Practice’, Journal of Agricultural Education and Extension, 13(3), pp. 209–225. Available at: https://oro.open.ac.uk/8581/.

Jones, J., Bosch, O., Drack, M., Horiuchi, Y. and Ramage, M. (2009) ‘On the design of systems-oriented university curricula’, The Research Reports of Shibaura Institute of Technology (Social Sciences and Humanities), 43(1), pp. 121–130. Available at: https://oro.open.ac.uk/21827/.

Moon, K. and Blackman, D. (2017) A guide to ontology, epistemology, and philosophical perspectives for interdisciplinary researchers, Integration and Implementation Insights. Available at: https://i2insights.org/2017/05/02/philosophy-for-interdisciplinarity/.

OECD (1986), OECD Economic Surveys: United Kingdom 1986. OECD Publishing, Paris. Available at: https://doi.org/10.1787/eco_surveys-gbr-1986-en.

Reynolds, M., Gates, E., Hummelbrunner, R., Marra, M., and Williams, B. (2016) Towards Systemic Evaluation. Syst. Res, 33: 662–673. Available at: https://doi.org/10.1002/sres.2423.

SLIM (2004) The role of conducive policies for fostering social learning for integrated management of water. SLIM, Open University: Milton Keynes. Available at: http://slim.open.ac.uk.

Sterling, S. (2004) ‘Higher Education, Sustainability, and the Role of Systemic Learning’, in P.B. Corcoran and A.E.J. Wals (eds) Higher Education and the Challenge of Sustainability: Problematics, Promise, and Practice. Dordrecht: Springer Netherlands, pp. 49–70. Available at: https://doi.org/10.1007/0-306-48515-X_5.

The Open University (no date) TB872 Part 2. Available at: https://learn2.open.ac.uk/mod/oucontent/view.php?id=2303521.

Wadsworth, Y. (1997). Everyday Evaluation on the Run. Routledge. Available at: https://doi.org/10.4324/9781315428857.

Williams, B (2009) Systems and evaluation. Available at: http://www.bobwilliams.co.nz/Systems_Resources.html.