logo-dw

Go Back   Dreamweaver Club Forums > Dreamweaver forums > SEO
Register FAQ Members List Search Today's Posts Mark Forums Read

Closed Thread
 
Thread Tools Display Modes
Old 07-07-2013, 02:47 PM   #1
vinnyvangogh
 
Join Date: Aug 2009
Posts: 84
Default Robots blocking folders

Hi,

I have a website which has folders which are not needed and folders that are needed - but include dead pages.
I have "disallowed" using robots.txt to stop Google indexing them.
Question is having a site map - which does not show the folders - when is it safe - from a GOOGLE indexing and busted links point of view -to just delete them and change the robots.txt file
vinnyvangogh is offline  
Old 07-08-2013, 07:17 AM   #2
edbr
edbr's Avatar
 
Join Date: Aug 2005
Location: Bali
Posts: 11,176
Default

if they are not needed why keep them there just clear the site of the unneeded stuff
__________________
If you're happy and you know it shake your meds!
different style links examples

Flight / Hotel search
Free script download
Bali Villas
edbr is offline  
Old 07-08-2013, 03:04 PM   #3
vinnyvangogh
 
Join Date: Aug 2009
Posts: 84
Default

Quote:
Originally Posted by edbr View Post
if they are not needed why keep them there just clear the site of the unneeded stuff
The blurb I read suggests that just deleting pages and folder - which are indexed with Google et al.. are seen as bad links /404s/301s etc and rack up bad points which can affect Googles listing status.

I presumed that the robots blocking will lead to the SEs indexing only whats allowed and eventually the blocked stuff can be deleted.. without adverse affect.
vinnyvangogh is offline  
Old 07-09-2013, 02:28 AM   #4
edbr
edbr's Avatar
 
Join Date: Aug 2005
Location: Bali
Posts: 11,176
Default

i read that to but am not convinced its a huge factor. 404 stay on google records but you can apply to get them delisted. i dont see the blocking will make that much difference. sites change naturally so if the page is important set a redirect permanent
__________________
If you're happy and you know it shake your meds!
different style links examples

Flight / Hotel search
Free script download
Bali Villas
edbr is offline  
Old 07-15-2013, 01:21 AM   #5
vinnyvangogh
 
Join Date: Aug 2009
Posts: 84
Default

Quote:
Originally Posted by edbr View Post
i read that to but am not convinced its a huge factor. 404 stay on google records but you can apply to get them delisted. i dont see the blocking will make that much difference. sites change naturally so if the page is important set a redirect permanent
I decided to delete everything not wanted on the site in the rebuild.
No more need for robots.txt file... renewed the site map and re-submitted for a re-index.
I do not expect much from Google for a few weeks... but so far its looking OK apart from busted links looking for the content I deleted - which presumably will go away when the site is actually fully reindexed.
vinnyvangogh is offline  
Old 07-15-2013, 02:17 AM   #6
edbr
edbr's Avatar
 
Join Date: Aug 2005
Location: Bali
Posts: 11,176
Default

that may take sometiime which is part of my possibly twisted logic that google dont give it a of of credence . they do say somewhere that they expect 404 's
__________________
If you're happy and you know it shake your meds!
different style links examples

Flight / Hotel search
Free script download
Bali Villas
edbr is offline  
Closed Thread


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT. The time now is 02:06 PM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Copyright 2006 DreamweaverClub.com