OK, I promise I’ll shut up after this, but PLEASE don’t let yourself think of this primarily in terms of CSAM. That’s the example that people who want to ban things always lead with, but it is a bad, atypical example, and you’ll get in trouble if you treat it as the paradigm.
It’s a bad example precisely because there’s a very strong consensus it shouldn’t be there, a relatively strong consensus about what it is, and various semi-independent banning agencies in that space are relatively credible. I’m not saying that I believe NCMEC or IWF or whoever should actually get the kind of power they have. Even if they were absolutely perfect and incorruptible, that would be too much trust. But they do have a fair amount of credibility, especially compared to some of the more questionable government agencies of the world. Insofar as it can be determined, they don’t seem to have deliberately abused their power very much. Most people are willing to give them a fair amount of trust.
Nonetheless, even in that space, there are strong political forces that want to expand definitions. I believe that the desire to expand is a major, if unacknowledged, reason for people pushing to change the very name from the relatively unambiguous “child porn” to the potentially boundless “CSAM”. If I remember right, the UK has a relatively broad definition. There are political forces, primarily in the US, who are trying to brand any child-directed media that acknowledge the existence of LGBTQSMNOP people as “sexual grooming”, and you’d better bet that, if they got any mainstream traction, they would try to get that material included in various definitions of CSAM.
So that pretty consensus is subject to evaporating any time.
As soon as you go beyond “CSAM”, even as far as “terrorist content”, let alone copyright, miltary secrets, drug information, hate speech, Moral Corruption™, Winnie the Pooh, Gollum, or whatever, the consensus doesn’t exist to begin with. The international network of institutions isn’t there. The government interests change, including various governments coming into direct conflict. But the pressure is, or will be, or could be, even greater, in some places. If you don’t have an answer for those harder issues, you probably don’t have an answer that can hold together for the long term.
You’re trying to build a network with a long life. You can’t assume that an approach designed for one relatively easy case (CP/CSAM), on easy mode (at a time when there’s very broad consensus on what it is and what should be done with it), with an easy response (the answer is always an absolute ban), will help you much in the long term.
I also still don’t believe it will work even for that case, mind you…