What to do about this inexplicable setter behaviour during JSON deserialize?

I'm trying to deserialize an API response into Apex classes using JSON.deserialize. Everything works fine, until I try to customise a setter on a list property. I've distilled this down to an anonymous Apex reproduction case.
class Child { String name {get; set;} } class Parent { String name {get; set;} List children { get; set { children = value; this.childCount = children.size(); } } Integer childCount {get; set;} } String jsonStr = '{"name": "Alice", "children": [{"name": "Bobby"},{"name": "Charlie"}]}'; Parent p = (Parent)JSON.deserialize(jsonStr, Parent.class); System.debug('p.childCount: ' + p.childCount); System.debug('p: ' + JSON.serialize(p)); System.debug('p.children.size(): ' + p.children.size());
This produces the debug output:
|DEBUG|p.childCount: 0 |DEBUG|p: {"name":"Alice","children":[{"name":"Bobby"},{"name":"Charie"}],"childCount":0} |DEBUG|p.children.size(): 2
Note that childCount is 0, but children is set correctly. I can even get the correct count after the deserialization is complete. That doesn't make sense, so I added some debugging to my setter:
set { System.debug('value: ' + JSON.serialize(value)); children = value; System.debug('children.size(): ' + children.size()); System.debug('children.clone().size(): ' + children.clone().size()); this.childCount = children.size(); }
this adds the debug output:

|DEBUG|value: [] |DEBUG|children.size(): 0 |DEBUG|children.clone().size(): 0
It appears that value is empty? And even if value has some special compiler-magic that prevents it from being inspected, children are also empty after the assignment. But I get the debugger output, so I know my setter is being called.
Finally, I tried using a new List to avoid any "specialness" around value:
set { children = new List(); children.addAll(value); System.debug('children.size(): ' + children.size()); this.childCount = children.size(); }
And now I don't even get the children:
|DEBUG|children.size(): 0 |DEBUG|p.childCount: 0 |DEBUG|p: {"name":"Alice","children":[],"childCount":0} |DEBUG|p.children.size(): 0
I have also tried writing an explicit getter, and using "this.children" instead of "children" in the setter, with no difference in results. What is happening?

Why do I care? The actual object model is much deeper, and the real "Parent" object is well down the hierarchy, and may have anywhere from 1 to thousands of children. I was trying to add a simple setter that captures the list size ("childCount"), and then clears the list if it has more than 10 members. I could iterate the whole deserialized structure, but a setter seems like a more efficient way to take action based on the contents of the property.