All Packages  Class Hierarchy  This Package  Previous  Next  Index

Class Webcrawler.Controller.Grabber

java.lang.Object
   |
   +----Webcrawler.Controller.Controller
           |
           +----Webcrawler.Controller.Grabber

public class Grabber
extends Controller
implements ActionListener
This is the Controller for grabbing a whole site. It extends the superclass Controller by adding a new tab to the window where the user can enter the local destination of the grabbed site, the maximum size of a file and whether pictures, sounds and other loadable links should be loaded as well. The local files will be stored at the specified destination in subdirs like host+directories and the same filename as on the internet. All local files stay on the harddrive for offline-browsing. The Grabber also rewrites the downloaded HTML-files so that links that previously pointed into the net, now point to the local harddrive.


Variable Index

 o allnodes
 o destDirField
 o filesToRename
 o HTMLfiles
 o lfnum
 o loadAllLinksBox
 o maxContentLenField
 o readyToRename
 o replace

Constructor Index

 o Grabber()
Creates a new Controller-window containing a new tab called "Grabber" which lets the user enter the local destination, the max.

Method Index

 o actionPerformed(ActionEvent)
Catches the ActionEvents when one of the buttons in the "General"-tab was pressed always call super.actionEvent(e) in subclasses at the end of the subclass' method
 o checkHPF(String)
 o copyFile(String, String)
 o crawlerDone()
When the crawler is done, this controller has to replace links in the locally stored HTML-files, so that they can be used for offline-browsing.
 o everySecond()
After all files have been downloaded and stored locally, the crawlerDone-method calls the replaceLinks()-method which creates temporary files (rewritten for offline-browsing).
 o finishGrabber()
 o loadCurrSettings()
This method is called from the superclass' actionPerformed method.
 o loadLink(LoadableNode)
Loads all HTML-files and, if the user selected "Load all links", all other loadable links as well.
 o localFileName(LoadableNode)
Basicly returns destination+host+file.
 o moveToRightPlace(LoadableNode)
 o nodeDone(URLNode)
Adds the node to the list of HTML-nodes that need to be rewritten.
 o replaceLinks(String)
Parses through the local HTML-file and creates a temporary file where all the links are rewritten to point to locally stored files.
 o saveCurrSettings()
This method is called from the superclass' actionPerformed method.
 o updateLists(String, String)

Variables

 o destDirField
 protected JTextField destDirField
 o maxContentLenField
 protected JTextField maxContentLenField
 o loadAllLinksBox
 protected JCheckBox loadAllLinksBox
 o lfnum
 private int lfnum
 o allnodes
 private Vector allnodes
 o replace
 private Vector replace
 o HTMLfiles
 private Vector HTMLfiles
 o filesToRename
 private Vector filesToRename
 o readyToRename
 private boolean readyToRename

Constructors

 o Grabber
 public Grabber()
Creates a new Controller-window containing a new tab called "Grabber" which lets the user enter the local destination, the max. size of the files and whether all other kinds of loadable links should be downloaded as well.

Methods

 o actionPerformed
 public void actionPerformed(ActionEvent e)
Catches the ActionEvents when one of the buttons in the "General"-tab was pressed always call super.actionEvent(e) in subclasses at the end of the subclass' method

Overrides:
actionPerformed in class Controller
 o saveCurrSettings
 protected void saveCurrSettings()
This method is called from the superclass' actionPerformed method. Here the entries of the StringFinder-tab are stored to set\strfnd.dat

Overrides:
saveCurrSettings in class Controller
 o loadCurrSettings
 protected void loadCurrSettings()
This method is called from the superclass' actionPerformed method. Here the entries of the StringFinder-tab are loaded from set\strfnd.dat

Overrides:
loadCurrSettings in class Controller
 o everySecond
 protected void everySecond()
After all files have been downloaded and stored locally, the crawlerDone-method calls the replaceLinks()-method which creates temporary files (rewritten for offline-browsing). The original files need to be deleted and the temporary files renamed to the original file's name. Since the system (e.g.Windows) only releases file ressources after a few seconds, the files can't be deleted in replaceLinks() right away. The filenames that need to be replaced by temporary files are written into filesToRename instead, and this method keeps trying to do the deleting/renameing until no more temporary files exist (=filesToRename is empty).

Overrides:
everySecond in class Controller
 o loadLink
 public boolean loadLink(LoadableNode n)
Loads all HTML-files and, if the user selected "Load all links", all other loadable links as well.

Returns:
Shall the specified node be loaded true/false
Overrides:
loadLink in class Controller
 o localFileName
 public String localFileName(LoadableNode n)
Basicly returns destination+host+file. All the local-file-names are also stored in the replace-field, so that the replaceLinks()-method can easily rewrite the downloaded HTML-files.

Overrides:
localFileName in class Controller
 o nodeDone
 public void nodeDone(URLNode n)
Adds the node to the list of HTML-nodes that need to be rewritten.

Overrides:
nodeDone in class Controller
 o crawlerDone
 public void crawlerDone()
When the crawler is done, this controller has to replace links in the locally stored HTML-files, so that they can be used for offline-browsing. This method calls replaceLinks() for all the downloaded HTMLFiles.

Overrides:
crawlerDone in class Controller
See Also:
replaceLinks
 o finishGrabber
 private void finishGrabber()
 o moveToRightPlace
 private String moveToRightPlace(LoadableNode n)
Returns:
the new localfile-name after copying the file
 o checkHPF
 private String checkHPF(String hpf)
Returns:
a new hpf-String that might be different now
 o updateLists
 private void updateLists(String oldlocfil,
                          String newlocfil)
 o copyFile
 private void copyFile(String oldf,
                       String newf)
 o replaceLinks
 private void replaceLinks(String filename)
Parses through the local HTML-file and creates a temporary file where all the links are rewritten to point to locally stored files. The filename is then added to the filesToRename-Vector so that the everySecond()-method can delete the originals and rename the temporary files to their original name.


All Packages  Class Hierarchy  This Package  Previous  Next  Index