Technology

No computer I know of ever came with an actual hardware reference.

However the concept is shown for instance on page 60, that shows peek and poke address for border and background color.
The C64 had a lot to get you started, way more than most, but it is still just to get you started.
If you want to get serious on a C64 you don't peek and poke much, but program in assembly.

but the reality was when you wanted to get the low level hardware, there was very little documentation.

The low level of what? Graphics sprites sound and IO were ALL documented!
Maybe horizontal and vertical smooth scrolling wasn't in the included book, which is essentially just an introduction for beginners, but such things were absolutely released info by Commodore, and it was dead easy to do for that reason, such info was everywhere!

I don’t sense that you have any actual experience programming on that platform,

I absolutely did, and I programmed sprites in assembly, and made a program we called sprite design, where you could design and animate sprites, which we never released, because we were under the false assumption that you didn't release software until it was perfect.
Later when i didn't use the C64 anymore, a friend of mine borrowed all my software, and came back absolutely ecstatic about how professional Sprite Design was, and was very puzzled he had never heard about it.
We made a build in help function using our own 90% efficient compression, we used self modifying code, and utilized the 6510 ability to switch off the ROM to have access to the RAM at that address space, and swapped where the character set was located and used our own 6 pixel wide character set, with an interrupt to give a tiny beep sound with key presses. The main structure was made with the Petspeed compiler, but everything surrounding the sprite animations was assembly. ( fuck 8 bit programming 😜 ) I made pretty sophisticated algorithms to make the weird 8x8 or 4x8 graphics format in color easier and faster to work with.
The C64 was amazing for its time for its speed and hardware capabilities. Despite being a machine that ran slightly below 1 MHz it was quite fast for its time.

You just probably isn't aware that all that was openly available on the C64 wasn't on most other computers of the time.

A collection on C64 books: https://archive.org/details/commodore_c64_books

An example of a book describing assembly and hardware registers:
https://archive.org/details/Assembly_Language_Programming_With_the_Commodore_64_1984_Brady_Communications_Company/mode/2up

But also there was a ton of info released in magazines like RUN etc.

I'm not sure what info exactly you think was lacking? Except of course there were a few things that were possible that even the creators of the chips were unaware of, But was figured out by hackers. Such things can obviously not be part of the official documentation.