Choice isn’t a reinforcer. So why is the behavior increasing?

A while ago, a post on one of my favorite blogs motivated me to write more on something I’ve been wanting to write on for a long time–choice, and how offering Nala the opportunity to make choices has increased her confidence and comfort out in the world. In fact, offering Nala choices has caused a lot of behaviors to increase. But does that mean that choice is a reinforcer?

Eileen is right. That’s just not how the quadrants work. 

If you’re actually offering choice as a consequence–which is where positive reinforcement happens–then, Eileen says, it looks like this:

Let’s say I’m going to ask my dog Clara to perform a sit. A couple of feet away from her I have her red rubber ball and a favorite toy, with a little space between them.

• Antecedent: I cue a sit
• Behavior: Clara sits
• Consequence: Whichever thing Clara grabs after I mark the sit: ball or toy

What role would the choice after the behavior perform? In this case, I believe very little. The ball and the toy would each have a much stronger and more direct effect than the fact that the dog got to choose. The choice aspect would likely be overwhelmed.

I agree with Eileen: if you actually offer choices as a reinforcer, it’s impossible for most of us to assert that the ability to make a choice, and not the toy or food that our dog chose, was the thing that caused the behavior to increase.

To be perfectly honest, I’ve noticed, as Nala and I have grown together, that it is usually extremely difficult to make a judgment about whether reinforcement is happening in real time at all–I can’t know whether a behavior is increasing or decreasing until we’ve done some trials, and measured it overall, in lots of situations

This fox squeaks loudly when chomped. Nala likes things that squeak when she murders them. She probably cares more about the squeakers than about the toys she left on the ground.

Similarly, I’m pretty sure that offering Nala a choice of toys as reinforcers would just not make sense to her. She would either be confused by the presentation of multiple choices and walk away, or snatch the first thing she encountered and shove it at me (while squeaking it, if it happened to have a squeaker).In fact, once I did exactly that–I brought Nala outside to play with me with a selection of toys on the ground to see what she would do. She picked the squeaky one. She likes squeakers. She’s a simple creature, in that way.

The choice itself would be–and, in that case, probably was–totally insignificant to her. I try to offer reinforcement opportunities that I think she will enjoy, and I take note of her preferences and allow her to pursue them (for instance, Nala will play fetch for longer if I let her chew the ball whenever she wants to instead of demanding perfect returns to hand on every repetition).  I can’t just offer Nala a  bunch of things and let her choose her reinforcer. To be an effective motivational trainer, I have to carefully observe her, and make intelligent choices about what reinforcers I offer at various times. There are no shortcuts.

On the other hand, one of Nala’s favorite things to do is to find a perfectly perfect stick and invite me to play with her. I have to wonder whether the fact that playing with that particular stick was her idea increases her enthusiasm for the game. And whereas toy play can be hit-or-miss for Nala, especially outside of our house and yard, I can always feel sure that she’ll play with a stick that she finds and brings to me on a hike. Which brings me to my next point:

Good Things Happen When Nala Gets to Make Choices

I don’t think I’ve ever claimed that choice reinforced a behavior.  But I think I can imagine why someone would say such a thing, and I hope you’ll indulge me as I think aloud about it.

A little over a year ago, I realized that Nala was uncomfortable much of the time when we were outside–but I just couldn’t figure out why. All I knew was that she pulled a lot, and she sniffed a lot, and her body language just looked sort of upset and stiff. And when we would drive to a brand new place for a hike, she wouldn’t be able to take food for several minutes–a big red flag with my dog who will try to eat even when she’s too freaked out to swallow her treat.

I’m an overthinker, and whenever I’m faced with a problem I attack it from as many angles as possible. So, one of the things that I did after that terrible hike where Nala wouldn’t even eat was buy her a new, better fitting harness, and start walking her on a long line instead of a leash whenever it was safe to do so. And I saw an immediate improvement. Nala’s recovery time in new places went down from half an hour to just a few minutes, and she moved more loosely, more bouncily, and with her tail a little higher, a little wavier, a little bit more confident, than she ever had in an unfamiliar place before. All thanks to comfier equipment that lets her make more choices about what to do and where to sniff.

And as hard as it can sometimes be to decide whether my reinforcement choices will work long-term in the middle of a training session, it has never been hard for me to see in real time how much better it is to let Nala vote with her feet as we train. The first time I tried to teach her a chin target, I kept reaching toward her face, cupping her chin, and feeding her. Yikes! As you can probably imagine, that method got us exactly nowhere.

When I switched to shaping, and letting Nala walk around and decide how much she wanted to shove her face into my hand, we got the behavior started with some duration in less than a minute of training. Now, Nala frequently presses her chin into my hand as I pet her to show her affection, and she will even play the chin-rest game when I’m standing and facing her as we work outside.

Free shaping has helped Nala worry less about new objects in places where they don’t belong, too. For too long, when I would present a new object, I would try to lure and cajole Nala into interacting with it. Even though she eventually would, most of the time, the next time I brought something out it would be the same story–she was no more willing to interact with it on her own than she had been before.  Now that a weird object usually predicts free shaping, though, Nala rarely goes through that initial period of avoiding the object and needing to be clicked/treated for looking at it or being in its presence; instead, she immediately starts interacting with it (usually by bopping it with her paws or trying to climb it. Hey, we all have preferences). And she’s becoming less and less worried about novelty, as a by-product of this: it’s been a long time since she startled at a weird shaped rock on a hike, and whenever she sniffs the grocery bags, her tail wags in a higher, looser way than it used to–checking out the new things in her house has become an enjoyable hobby instead of a worried necessity.

As Eileen says in her article, all of these opportunities for Nala to make choices–primarily, choices about when, where, and how she moves her body around, above and beyond the freedom to walk away if she doesn’t want to play at all–occur before the behavior that then gets reinforced. That makes them part of the antecedents, or setting factors, for these behaviors (well, except for the long line and harness. I’m not sure how those factor into the ABC’s).  Choice definitely isn’t a reinforcer: it’s an aspect of the setting, and an integral part of our training, but behavior isn’t being reinforced by choice.

Then why are the behaviors I want–confidence in the world, checking in with me frequently, resting her chin in my hands, and interacting with objects–increasing? Isn’t that what reinforcement means–that behavior increases?

So Why Is The Behavior Increasing? 

I think that in the positive reinforcement scenarios that I describe above, something distinct from negative reinforcement is happening.

For one thing, when I write out the ABC’s of what happens when Nala and I do a chin rest training session, there is more at work than just pressure on/pressure off:

Antecedent: I hold out my hand
Behavior: Nala walks over and shoves her chin into my outstretched palm
Consequence: Nala gets one to five treats in my hand and a second treat thrown behind her to set up the next rep.
Prediction: Approaching me and shoving her chin into my hand should increase.

That’s a positive reinforcement scenario–although, admittedly, the tossed treat does relieve pressure and give her a chance to get away.

The factors that make this training scenario-which is, in many ways, a desensitization and counter-conditioning scenario, since Nala is sensitive to body pressure–work are happening even before the obvious antecedent above, I think.

I recently attended one day of a seminar given by Suzanne Clothier, where she gave some version of this speech at least once:

Like Clothier says, all kinds of things can compromise a dog’s feeling of safety. Something they are scared of being too close. Feeling trapped, as by a four to six foot leash, or restrained, as by an ill-fitting harness. Pressure from you.

Getting to Choose Can Help Dogs Feel Safe

If you are trying to help your dog overcome her worry about something, or trying to teach a behavior that is hard for her for some reason, it can be so easy to push too hard, too far, for too long. To get closer to the thing, to stay close to the thing for longer. We may know where our dog’s threshold usually is, or remember how close we got to the scary thing yesterday, and want to start there again. We know where she can eat, so we want her to eat there today so that we can start closer still tomorrow.

We have the best intentions. We want her to get better, and we know how this process works: we get closer and closer to the scary thing with no reactions, and our dogs get lots of awesome things for being near the scary thing, and eventually she stops being scared of it and her world gets to be bigger and better. We love her, and we want that day where she isn’t scared anymore to come as soon as possible.

But sometimes, if we set the pace for every interaction–especially every interaction that involves something our dogs find worrisome–we forget that behavior isn’t linear, and dogs have bad days, and triggers stack, and today may be different than yesterday. We get too close to the scary.  We forget to listen, and we push too hard for our dogs that day.

It’s okay. We’re all human, and we make mistakes, and dogs–even delicate, sensitive flowers like Nala–are amazingly resilient.

Shouldering the entire burden of managing your dog’s threshold is stressful and difficult and we’re bound to screw it up sometimes. But when we let our dogs make choices about how close they get, when they get closer, and how long they stay, that burden can be a little bit easier, since our dogs’ choices will let us know how safe they feel–we can let her choices, her feeling of safety, dictate the pace of desensitization. When they feel safe, they can learn: there’s room for operant and classical conditioning to work.

Of course, when we’re working with worried or frightened dogs, we need to make sure that our dogs don’t get themselves in over their heads and get into trouble. How we can do those things is well outside the scope of this post.

Conclusion?

I’m definitely not using choice as reinforcement, but I can see why people might be tempted to say that choice is reinforcing for dogs. After all, I’ve seen for myself how Nala brightens, how proud and how full of herself she looks, when she chooses to be brave.

When I give Nala choices, she is less worried in the world, more confident, and more likely to remember my existence and turn to me for companionship or reassurance than she used to be–but not because choice positively reinforced her. It’s because giving her choices reduces the pressure in those situations, reduces her stress and frustration. It lets her act according to how safe she feels–and the fact that I don’t push her past the point of safety helps her learn that she can trust me.

And I didn’t even manage to touch on my wild speculations about the mental lives of dogs, and how much they enjoy their own ideas! I might have even more to say on this!

More Resources and Examples: 

Eileen Anderson has many great posts that demonstrate how good desensitization and counterconditioning plans assure the student’s feeling of safety by staying way, way under threshold. This one is about Clara:
Socializing a Formerly Feral Dog

This thoughtful post and video by Sonya Bevan demonstrate the power of letting a dog choose to engage in a simple husbandry task:
The Saga of Measuring a Dog for a Muzzle: Tolerating vs Enjoying Handling

And if you would like weeks upon weeks of information on this and related topics, Amy Cook’s Dealing with the Bogeyman class at Fenzi Dog Sports Academy might be a good fit for you. It has certainly helped Nala and me:
Dealing With the Bogeyman: Helping Fearful, Reactive, and Stressed Dogs

Do you have a dog who worries? What happens when you let your dogs decide what to do?