I just wanted to log this in case anyone else comes across it. I just did a fresh install of Windows Server 2019 and SQL Express 2019. After following the install instructions for installing the SQL server here (https://kb.sambapos.com/en/2-1-3-a-sql-express-server-setup/) it was still denying a connection on the client. This was validated by disabling Windows Defender after which the client computer connected fine.
After looking through the firewall settings it appears that even though the firewall ports were allowed for 1433 it still couldn’t establish a connection since SQL Instance will be on a random/dynamic port which the SQL Browser will direct the connection to (https://superuser.com/questions/1303512/unblocking-windows-10-defender-firewall-for-ms-sql-server-2014). After adding both SQL Server and SQL Browser executables (which can be identified in the server’s services directory) the connection was fine. I didn’t have this issue with the installation with previous versions of Windows.