Per 1.7 spec 3.8.1, there are multiple logical text string types:
* text strings
* ASCII strings
* byte strings
Text strings can be in UTF-16BE, PDFDocEncoding, or (since PDF 2.0)
UTF-8.
But byte strings shouldn't be converted but treated as binary
data.
This makes us no longer convert strings used for drawing page text.
TABLE 5.6 "Text-showing operators" lists the operands for text-showing
operators as just "string", not "text string" (even though these strings
confusingly are called "text strings" in the body text), so not doing
this there is correct (and matches other viewers).
We also no longer incorrectly convert strings used for cypto data
(such as passwords), if they start with an UTF-16BE or UTF-8 marker.
No behavior change for outlines and info dict entries.
https://pdfa.org/understanding-utf-8-in-pdf-2-0/ has a good overview of
this.
(ASCII strings only contain ASCII characters and behave the same
anyways.)
e39a790c82