# Why Aren’t Thoughts, Feelings, and Behavior Changing?

A PE ([see primer here](/level-linkages/linkage-across-levels/process-evaluations.md)) linked to a level 3 evaluation can surface what is or isn’t enabling cognitive, affective, or behavioral outcomes, informing what to test and where to focus program improvements.

At this level it is important to zoom out from the user into the broader program ToC (Figure 3 in Building Blocks section). Behavior change is shaped by the program delivery system and social context outside the program: the workflows, organizational, and social conditions that determine whether product use translates into changed thoughts, feelings, and behavior. 

| ToC Domain / Assumption | Example PE questions                                                                                                                                   | Text                                                                                                                                                                                           |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Interpretation          | Are users receiving and interpreting AI outputs in the way the program intended, and does this differ across subgroups?                                | Semi-structured interviews; cognitive walk-throughs with purposive sample of users; analysis of in-conversation signals (e.g., follow-up questions, expressed confusion), quantitative surveys |
| Opportunity to act      | What contextual factors- social norms, competing demands, or structural constraints- are enabling or blocking users from acting on AI recommendations? | Focus groups and ethnographic observation; barrier/enabler mapping using implementation science frameworks                                                                                     |
| Externalities           | Has the tool shifted the roles or behaviors of others in the program ecosystem — intentionally or not? Are these shifts undermining outcomes?          | Key informant interviews and/or surveys with supervisors and non-user staff; focus groups; administrative data on workload, staffing, or service utilization                                   |

A PE at Level 3 typically requires richer qualitative inquiry than at Level 2, since barriers to behavior change are often rooted in context, relationships, and norms. When possible, run the PE alongside or before the Level 3 evaluation, so findings can directly inform program refinements.

<details>

<summary>💬 Want to suggest edits or provide feedback?</summary>

{% embed url="<https://tally.so/r/A788l0?originPage=level-3-user-evaluation%2Fhow-is-level-3-evaluation-performed%2Fuser-privacy-and-security>" %}

</details>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://eval.playbook.org.ai/user-expereince/how-to-evaluate/user-privacy-and-security.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.