I can see the value of both approaches. (And I work for an auth provider, FusionAuth.) It's the age-old conflict between "use the best tool for the job" and "YAGNI".There's something so simple about having a single deployment of code and auth wrapped up together, which is what the library approach offers. You can use the database to maintain integrity between your app's domain objects and your users. Everything needed for your app is in one place. I've seen this succeed and been part of teams doing this myself.
But auth/user management is usually the first thing extracted when you start to grow for a reason. Otherwise you end up with your main application serving as both its main purpose and as a user data store for other apps, or worse, you have siloed user data, multiple profiles and credentials for customers, and a worse user experience because your users have to log in to each app separately.
The extraction of this functionality is not as simple as moving other services typically, because of availability reqs and password hashes (not every provider supports every kind of hash, though I will say I wish more folks implemented something like FusionAuth's approach[0]). So that's an argument to just start with a separate service.
And, as you say, a single source of truth of user data for multiple applications has surprising benefits in the long run which counteract the additional complexity. Offering single sign-on across multiple apps[1] is a great feature with user benefits, if you have or plan to have multiple applications. Outsourcing auth to a specialized piece of software lets you focus on your app (once you've integrated, of course, nothing is free) and offer relatively undifferentiated features like MFA, passkeys, or SAML integrations with configuration rather than coding.
I guess the answer is "it depends", as always. I'd consider future plans, number of apps expected, features needed, cost structure, and more when making this decision.
[0]: https://fusionauth.io/docs/extend/code/password-hashes/custo...
[1]: if you and they use a standard like SAML or OIDC, you can even offer single sign-on for commercial off the shelf tools