Announcement

Collapse
No announcement yet.

Googlebots are no longer able to crawl my site, what happened?

Collapse
This topic is closed.
X
This is a sticky topic.
X
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Googlebots are no longer able to crawl my site, what happened?

    This may be attributed to your MTU setting on your main eth interface (usually eth1 or eth0). The below example applies to an RH-based OS.

    When you are logged into shell as root or a privileged user, type in:

    ifconfig ethX mtu X

    Where ethX is either eth0, eth1, etc.. depending on your configuration and mtu X is a number in seconds. To see what you currently have it set at, you can run the ifconfig command all by itself and look at the MTU value returned. A good number for googlebot to properly crawl your site is around 500. So, as a full example, if your main interface is eth1, you'd run the following command from the prompt:

    ifconfig eth1 mtu 500


    To have this take effect on reboot/startup, add the following line to your /etc/rc.local file:

    /sbin/ifconfig eth1 mtu 500;
    www.AYKsolutions.com
    From Shared to Dedicated
    Professional. Painless. Polite.
Working...
X