https://adobe-type-tools.github.io/font-tech-notes/pdfs/T1_SPEC.pdf,
8.4 First Four Subrs Entries:
"""If Flex or hint replacement is used in a Type 1 font program, the
first four entries in the Subrs array in the Private dictionary must be
assigned charstrings that correspond to the following code sequences. If
neither Flex nor hint replacement is used in the font program, then this
requirement is removed, and the first Subrs entry may be a normal
charstring subroutine sequence. The first four Subrs entries contain:
Subrs entry number 0:
3 0 callothersubr pop pop setcurrentpoint return
"""
othersubr handler 0 gets three arguments:
* The flex height (the distance after which the bezier splines
are replaced with just straight lines)
* The current position after the flex
It pushes that position on the postscript stack, where predefined subr
handler number 0 then pops it from. It then passes it to
setcurrentpoint.
In theory, we now correctly do that setcurrentpoint call, which we
previously weren't.
In practice, that setcurrentpoint call always receives the last point of
the flex -- and our path api apparently gets confused when move_to() is
called on it when the current point is already at that same location.
So tweak the SetCurrentPoint handler to not set the current point on
the path if it's already the path's current point, with a FIXME to
figure out what exactly is happening in Gfx::Path.
No big behavior change if flex is used, but this is more correct if it
isn't.
(This only works because our `return` handler is empty, else we would
have to make the callothersubr handler start a call frame.)
https://adobe-type-tools.github.io/font-tech-notes/pdfs/T1_SPEC.pdf,
8.4 First Four Subrs Entries:
"""If Flex or hint replacement is used in a Type 1 font program, the
first four entries in the Subrs array in the Private dictionary must be
assigned charstrings that correspond to the following code sequences. If
neither Flex nor hint replacement is used in the font program, then this
requirement is removed, and the first Subrs entry may be a normal
charstring subroutine sequence. The first four Subrs entries contain:
[...]
Subrs entry number 1:
0 1 callothersubr return
Subrs entry number 2:
0 2 callothersubr return
"""
So subr entry numbers 1 and 2 just call othersubr 1 and and 2, which
means we can just move the handling code over.
No behavior change if flex is used, but more correct if it isn't.
(This only works because our `return` handler is empty, else we would
have to make the callothersubr handler start a call frame.)
Previously we assumed a default precision of 6, which made the printed
values quite odd in some cases.
This commit changes that default to print them with just enough
precision to produce the exact same float when roundtripped.
This commit adds some new tests that assert exact format outputs, which
have to be modified if we decide to change the default behaviour.
This is a subset of #21484: Type 2 CFFs never use the special subrs,
so stop doing them for type 2 at least for now.
Fixes an assert in 0000064.pdf in 0000.zip in the pdfa dataset
(a stack underflow because a subr is supposed to push a bunch of
stuff, but instead it ran one of the built-in routines instead of
the subr from the font file).
As discussed in #21484, this isn't right for type 1 CFFs either,
but just removing the code there regresses Tests/LibPDF/type1.pdf.
A slightly more involved thing is needed there; I added a FIXME
for that here.
Previously, an xref stream with a field with larger than 8 would
result in an undefined shift occurring. We now ensure that each field
width is a number and is less than or equal to 8.
Nothing in PDF 1.7 spec 8.2.1 Destinations mentions the page being
`null`, but it happens in 0000372.pdf (for the root outline element)
and in 0000776.pdf (for every outline element, which looks like a
bug in the generator maybe) of 0000.zip from the pdfa dataset.
Documents using them now show render-time diagnostics instead
of asserting that number of parameters passed to a color don't
match whatever number of channels the previously-set color space
had.
Fixes two asserts on the `-n 500` 0000.zip test set.
Same as other recent color spaces: Enough to make us not assert,
but not enough to actually produce color.
Fixes 2 asserts on the `-n 500` 0000.zip pdfa dataset.
0000101.pdf from 0000.zip from the pdfa dataset has /Height set to
an indirect object that contains an int.
Make that work, and make sure various other similar places getting
values of the image dict also resolve indirect references.
No intended behavior change.
It does have the effect that indirect object references now go down
the array path instead of the number path. They still fall over there,
but now that's easy to fix.
0000459.pdf in 0000.zip in the pdfa dataset contains this as the
very first object:
```
1 0 obj
<<
/Creator (Developer 2000)
/CreatorDate (
/Author (Oracle Reports)
/Producer (Oracle PDF driver)
/Title (2021_06_29 Tutoritzacions APTES.PDF)
>>
endobj
```
The `/CreatorDate` value string is unterminated.
Before, we'd assert when trying to check if the first object is
a linearization dict.
Now, we never read the first object (an error during the linearization
dict reading is treated as "file is not linearized") unless we try
to print the document's metadata -- and there we now show an error
instead of asserting.
Actually using separation color spaces still doesn't work, but we
now no longer assert on them when they're used.
Fixes 2 crashes on the `-n 500` 0000.zip pdfa dataset.
Several files have a comment after the trailer dict and the
`startxref` after it.
We really should add a consume_whitespace_and_comments() function
and call that in most places we currently call consume_whitespace().
But in this case, for non-linearized files, we first jump to the
end of the file, read `startxref`, then jump to `xref` from the
offset there, and then read the trailer after the `xref`,
only to read `startxref` again. So we can just not do that.
(For linearized files, we now completely ignore `startxref`.
But we don't use the data in `startxref` in linearized files
anyways, so it's fine to not read it there too.)
Reduces number of crashes on 300 random PDFs from the web (the first 300
from 0000.zip from
https://pdfa.org/new-large-scale-pdf-corpus-now-publicly-available/)
from 25 (8%) to 23 (7%).
A page's /Contents can be an array of streams, and the page's contents
are then as if those streams are concatenated.
Most of the time, a stream ends with whitespace. But in some cases
(e.g. 0000642.pdf from 0000.zip from the pdfa dataset), the first
stream ends with an operator (`Q`) and the next stream starts with
one (`q`), and the concatenation would form a new, unkonwn operator
(`Qq`). Separate the streams' contents with a space to prevent that.
Reduces numbers of PDF files we fail to open in the -n 500 case
from 11 to 10 (in either case, we then crash on 18 of the PDFs
that we do manage to open).
Type 1 fonts usually have a m_font_program and no m_font -- they only
have m_font if we're using a replacement font for the fonts that
were built-in to PDFs before Acrobat 4.0 (and must still work to
show existing files).
However, SimpleFont::get_glyph_width() used to always return a
float, which in Type1Font was only implemented if m_font was set.
Per spec, we're supposed to just use /MissingWidth for fonts that
are missing an entry in the descriptor's /Width array. However, for
built-in fonts, no explicit /Width array is needed (PDF 1.7 spec,
Appendix H.3, 5.5.1). So if we just always use /MissingWidth,
then PDFs that use a built-in font draw all their text on top
of each other (e.g. 000333.pdf from stillhq.com-pdfdb).
So change get_glyph_width() to return Optional<float>, return
it only in Type1Font if m_font is set, and use MissingWidth
if it isn't set.
That way, replacement fonts still return a width, and real
fonts that are supposed to have /Width and use /MissingWidth
for missing entries do what they're supposed to too, instead
of crashing.
From 20 (6%) to 16 (5%) crashes on the 300 first PDFs, and from
39 (7.8%) to 31 (6.2%) on the 500-random PDFs test.
`left` might be a number bigger than there are actually glyphs in the
CFF.
The spec says "The number of ranges is not explicitly specified in the
font. Instead, software utilizing this data simply processes ranges
until all glyphs in the font are covered." Apparently we have to check
for this within each range as well.
Needed for example in 0000054.pdf and 0000354.pdf in 0000.zip in the
pdfa dataset.
Together with the previous commit:
From 21 (7%) to 20 (6%) crashes on the 300 first PDFs, and from
41 (8.2%) to 39 (7.8%) on the 500-random PDFs test.
...and replace template instantiations with a loop, to make this
easily possible.
Vaguely nice for code size as well.
Needed for example in 0000054.pdf and 0000354.pdf in 0000.zip in the
pdfa dataset.
We used to use an u8 as loop counter, which would overflow
if there were more than 255 glyphs, producing hundreds of megabytes
of
Couldn't find string for SID x, going with space
output in the process, while all data until the end of the CFF
section got interpreted as SIDs, until a try_read() would finally
fail.
We now no longer fail miserably trying to render page 2 of
0000352.pdf of 0000.zip from the pdfa dataset.
Fixes just one crash of the larger 500-document test set, but
when I tweak test_pdf.py to print all stacks instead of just the
top 5, it no longer produces 260 MB of output.
Inline images can contain arbitrary binary data in the operator stream,
greatly confusing the operator parser.
Just skip them for now. They'll produce a
`Rendering of feature not supported: draw operation: inline_image_begin`
diag as usual, so we won't forget about it.
After #21536, reduces number of crashes on 300 random PDFs from the web
(the first 300 from 0000.zip from
https://pdfa.org/new-large-scale-pdf-corpus-now-publicly-available/)
from 23 (7%) to 22 (7%).
On a larger sample (`Meta/test_pdf.py -n 500 ~/Downloads/0000`),
reduces number of crashes from 53 (10.6%) with 36 distinct crash
stacks to 46 (9.2%) with 33 distinct stacks.
If a PDF uses `/CustomName cs` and `/CustomName` then points at just a
name like `/DeviceGray` instead of an array, that's ok. Just using
`/DeviceGray cs` is simpler, so this extra level of indirection is
somewhat rare in practice, but it's valid and it does happen. So support
it.
We already have a helper that does the right thing that we just need to
call.
Together with #21524 and #21525, reduces number of crashes on 300 random
PDFs from the web (the first 300 from 0000.zip from
https://pdfa.org/new-large-scale-pdf-corpus-now-publicly-available/)
from 29 (9%) to 25 (8%).
This fixes a small bug from 39b2eed3f6: That commit tried to disable
filters for the very first object read, for the case covered in
Tests/LibPDF/password-is-sup.pdf.
However, it accidentally also disabled filters by default.
Most of the time, this isn't really a difference: We call
`set_filters_enabled(true);` very early in
`DocumentParser::initialize_linearization_dict()`, which explicitly
enables filters, and `initialize_linearization_dict()` is the very
first thing called in `DocumentParser::initialize()`.
But there's an early exit in `initialize_linearization_dict()`
for if there's nothing looking like an indirect object right
after the header, and in this case we used to not enable
filtering, and would hand compressed streams to the operand parser.
(And due to a 2nd bug, we'd even do this if the header line was
followed by an empty line.)
0000990.pdf from 0000.zip from
https://pdfa.org/new-large-scale-pdf-corpus-now-publicly-available/
starts like so:
```
%PDF-1.7
4 0 obj
```
parse_heaader() used to put the cursor at the start of the 2nd,
empty, line. initialize_linearization_dict() would then check
if `m_reader.matches_number()` to see if there could possibly
be a linearization dict.
In this case, there isn't one, but we should detect linearization
dicts even if they're separated by whitespace from the first line.
Per spec:
"If the color space is one that can be specified by a name and no
additional parameters (DeviceGray, DeviceRGB, DeviceCMYK, and certain
cases of Pattern), the name may be specified directly."
We still don't implement /Pattern color spaces, but now we no longer
crash trying to look up the potentially-nonexistent /ColorSpace
dictionary on the page object when /Pattern is used directly as color
space name.
On top of #21514, reduces number of crashes on 300 random PDFs from the
web (the first 300 from 0000.zip from
https://pdfa.org/new-large-scale-pdf-corpus-now-publicly-available/)
from 42 (14%) to 34 (11%).
It used to be called ColorSpaceFamily::never_needs_parameters().
But in the cpp file, the macro arg was called ever_needs_parameters,
and the spec says
"If the color space is one that can be specified by a name and no
additional parameters (DeviceGray, DeviceRGB, DeviceCMYK, and certain
cases of Pattern), the name may be specified directly."
so let's use that language here.
No behavior change.
We now no longer crash on images that use an ICC-based color space.
Reduces number of crashes on 300 random PDFs from the web (the first 300
from 0000.zip from
https://pdfa.org/new-large-scale-pdf-corpus-now-publicly-available/)
from 81 (27%) to 64 (21%).
Also fixes all remaining crashes in
411_getting_started_with_instruments.pdf and
513_high_efficiency_image_file_format.pdf.
