Files using for this :
/etc/hosts.allow
/etc/hosts.deny
/etc/sshd.hosts
1) First add the ips that need to be whitelisted in the /etc/hosts.allow.
for that you need to add the ips to the file /etc/sshd.hosts file.
2) Add the below entry to the file /etc/hosts.allow.
sshd: /etc/sshd.hosts
3) Finally add the below entry to the file /etc/hosts.deny
sshd: ALL
This will deny ssh from all the locations except the ips given in the sshd.hosts file.
So if you want to whitelist anymore ips just add that to the file sshd.hosts.
Comments