The /etc/passwd file has to be world-readable for certain commands to run properly

Yep. Commands such as ls -l need to be able to read /etc/passwd. Part of what this command does is translate the UID for the owner of the file into a username. /etc/passwd is the only place on the system where that information is kept.

However, this means that any user on the system can look at the encrypted passwords if they are stored there.