After a little trial & error, I was finally able to get an SSH server running on Android through Termux. Some help from the following three articles as well:

You have to make sure you have the openssh package installed. If not, run pkg install openssh. Then start the service via sshd. Something I didn’t realize at first was that the default port was changed to 8022, instead of the standard 22. Fine by me, considering I wanted to change it from the default anyway.

Termux doesn’t support password authentication, & instead does it the more secure way, via keys. Below are the commands to get your keys setup properly.
touch ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
ssh-keygen # You can follow all of the defaults here, unless you want to have a passphrase
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

You then need to copy the id_rsa file (private key) to the device you want to connect from. To keep better track, I usually change the name of the file once it is on the other device. Keep in mind, the permissions on this file need to be changed as well. chmod 600 [file].

To connect, just run ssh [IP_Address] -p 8022 -i ~/path/to/private_key.