The evaluation, in detail

The Activat project has encouraged the overcoming of the community / hospital dichotomy towards a real evolution of the Catalan mental health system to a Recovery based system. The evaluation of the project, carried out through an experimental design with control group without intervention, was the subject of much debate between the participating entities. Apart from the design, another controversial aspect was the consideration of the circuit as a single block of intervention (understood as a public policy) in contrast to the possibility of evaluating each intervention separately (from a more psychosocial perspective) , which was ruled out. These two decisions (experimental design with inactive control group and consideration of all interventions as one block) were accompanied by the circumstance that the experimental evaluation was being carried out at the same time as the project’s own infrastructure was being implemented. That is, the need to incorporate participants in the randomization processes was given at the same time that the space technicians started their activity in the territories, psychoeducational programs were offered for the first time open to the public, the GAMs were starting to walk, etc.

From the Voices federation we have always argued that within the experimental design, as it should be given, the control arm was a waiting list that, once evaluated the impact on the intervention group, join the circuit. Not only would this have been more ethical, but it would also have enabled the control group to be exposed to the intervention at a later stage of implementation of the circuit structure.

The project has given us many lessons and has shown us the way to improve the circuit, making it more flexible, shortening some interventions that are perhaps too complex and, in general, facilitating the participation of people based on their needs. From the first person’s point of view, the main lesson we believe is that the participation of people with experience of psychosocial diversity should not be limited to activities, but should encourage participation at the technical level and management. The prioritization, for example of the internal versus external validity given in the evaluation of this project, has probably been due to the still sloping correlation of forces towards the prioritization of the external expert view versus the importance of the vision of the people involved.