If you want your Index server to crawl a particular WFE, leave the default setting alone and modify the HOSTS file yourself. This way you can direct it to hit one server for some sites and another for others.
The issue with the default behaviour of OOTB SharePoint feature to choose the WFE for the crawling is that it doesn't modify HOST file correctly.
SharePoint set the IP address of the WFE server to crawl in HOST file, but under some circumstances it may set the wrong IP. The reason for this is that it uses *only* the first NIC address (displayed when you do an IPCONFIG /ALL). But in case of SSL or multiple adapters you end up with the wrong IP. The situation is even worse when you try to modify that IP in HOST file manually, but it will be overridden with the old value in 5 mins :), because SharePoint uses timer jobs to update HOST file.
The solution is just don't rely on SharePoint "Central Administration > Operations > Services on Server > Office SharePoint Server Search Service Settings" actions and modify HOST file yourself.