org.wandora.application.tools.extractors.nyt

Class AbstractNYTExtractor

java.lang.Object

org.wandora.application.tools.AbstractWandoraTool

org.wandora.application.tools.extractors.AbstractExtractor

org.wandora.application.tools.extractors.nyt.AbstractNYTExtractor

All Implemented Interfaces:

java.io.Serializable, java.lang.Runnable, BrowserPluginExtractor, DropExtractor, WandoraTool, WandoraToolLogger, Handler, InterruptHandler, TopicMapLogger

Direct Known Subclasses:

NYTArticleSearchExtractor, NYTEventSearchExtractor

public abstract class AbstractNYTExtractor
extends AbstractExtractor

Author:

akivela, Eero Lehtonen

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private static class	AbstractNYTExtractor.DayOfWeek

Field Summary

Fields

Modifier and Type	Field and Description
(package private) static java.util.Map<java.lang.String,AbstractNYTExtractor.DayOfWeek>	abbrToDayOfWeek
static java.lang.String	ABSTRACT_SI
static java.lang.String	ARTICLE_SI
static java.lang.String	BODY_SI
static java.lang.String	BYLINE_SI
static java.lang.String	CLASSIFIER_FACET_SI
static java.lang.String	COLUMN_FACET_SI
private java.lang.String[]	contentTypes
static java.lang.String	DATE_SI
static java.lang.String	DBPEDIA_RESOURCE_SI
static java.lang.String	DES_FACET_SI
static java.lang.String	END_DATE_SI
static java.lang.String	EVENT_CATEGORY_SI
static java.lang.String	EVENT_DATE_SI
static java.lang.String	EVENT_DESCRIPTION_SI
static java.lang.String	EVENT_DETAIL_URL_SI
static java.lang.String	EVENT_LATITUDE_SI
static java.lang.String	EVENT_LONGITUDE_SI
static java.lang.String	EVENT_NAME_SI
static java.lang.String	EVENT_SI
static java.lang.String	EVENT_VENUE_SI
static java.lang.String	FACET_SI
static java.lang.String	GEO_FACET_SI
static java.lang.String	KEYWORD_SI
static java.lang.String	LANG_SI
static java.lang.String	LEAD_PARAGRAPH_SI
static java.lang.String	MATERIAL_TYPE_FACET_SI
static java.lang.String	NYT_SI
static java.lang.String	ORG_FACET_SI
static java.lang.String	PER_FACET_SI
static java.lang.String	RECURRING_DAY_SI
static java.lang.String	SOURCE_FACET_SI
static java.lang.String	START_DATE_SI
static java.lang.String	TEXT_SI
static java.lang.String	WEEKDAY_SI

Fields inherited from class org.wandora.application.tools.extractors.AbstractExtractor

CUSTOM_EXTRACTOR, DONE_FAILED, DONE_MANY, DONE_ONE, EXACTLY_GIVEN_URLS, FILE_EXTRACTOR, FILE_PATTERN, GIVEN_URLS_AND_ALL_CRAWLED_DOCUMENTS, GIVEN_URLS_AND_CRAWLED_DOCUMENTS_IN_URL_DOMAIN, GIVEN_URLS_AND_LINKED_DOCUMENTS, GIVEN_URLS_AND_URL_BELOW, INFO_WAIT_WHILE_WORKING, LOG_TITLE, POINT_START_URL_TEXT, RAW_EXTRACTOR, SELECT_DIALOG_TITLE, STRING_EXTRACTOR_NOT_SUPPORTED_MESSAGE, URL_EXTRACTOR

Fields inherited from interface org.wandora.application.WandoraToolLogger

CLOSE, EXECUTE, INVISIBLE, VISIBLE, WAIT

Fields inherited from interface org.wandora.application.tools.browserextractors.BrowserPluginExtractor

RETURN_ERROR, RETURN_INFO

Constructor Summary

Constructors

Constructor and Description
AbstractNYTExtractor()

Method Summary

Modifier and Type	Method and Description
static Topic	getAbstractTypeTopic(TopicMap tm)
static Topic	getArticleTypeTopic(TopicMap tm)
static Topic	getBodyTypeTopic(TopicMap tm)
static Topic	getBylineTopic(java.lang.String byline, TopicMap tm)
static Topic	getBylineTypeTopic(TopicMap tm)
static Topic	getCategoryTopic(java.lang.String category, TopicMap tm)
static Topic	getCategoryTypeTopic(TopicMap tm)
java.lang.String[]	getContentTypes() Returns an array of String containing the content-types this ContentHandler can process.
static Topic	getDateTypeTopic(TopicMap tm)
static Topic	getDayOfWeekTypeTopic(TopicMap tm)
static Topic	getDBpediaResourceTopic(java.lang.String res, TopicMap tm)
static Topic	getDBpediaResourceTypeTopic(TopicMap tm)
java.lang.String	getDescription() AdminToolManager views tool descriptions while user browses available tools and build user customizable GUI elements such as Tools menu.
static Topic	getDescriptionTypeTopic(TopicMap tm)
static Topic	getEndDateTypeTopic(TopicMap tm)
static Topic	getEventDateTypeTopic(TopicMap tm)
static Topic	getEventTypeTopic(TopicMap tm)
javax.swing.Icon	getIcon() All tools may have identifying graphic icon used within tool GUI elements.
static Topic	getKeywordNameTopic(java.lang.String name, TopicMap tm)
static Topic	getKeywordTopic(java.lang.String keyword, TopicMap tm)
static Topic	getKeywordTypeTopic(TopicMap tm)
protected static Topic	getLangTopic(TopicMap tm)
static Topic	getLatitudeTypeTopic(TopicMap tm)
static Topic	getLeadParagraphTypeTopic(TopicMap tm)
static Topic	getLongitudeTypeTopic(TopicMap tm)
java.lang.String	getName() Tools name represent the tool in UI unless the tool has been given explicitly another GUI name.
static Topic	getNYTTypeTopic(TopicMap tm)
protected static Topic	getOrCreateTopic(TopicMap tm, java.lang.String si)
protected static Topic	getOrCreateTopic(TopicMap tm, java.lang.String si, java.lang.String bn)
static Topic	getRecurringDayTopic(TopicMap tm, java.lang.String abbr)
static Topic	getRecurringDayTypeTopic(TopicMap tm)
static Topic	getStartDateTypeTopic(TopicMap tm)
static Topic	getTextTypeTopic(TopicMap tm)
static Topic	getVenueTopic(java.lang.String venue, TopicMap tm)
static Topic	getVenueTypeTopic(TopicMap tm)
protected static Topic	getWandoraClassTopic(TopicMap tm)
protected static void	makeSubclassOf(TopicMap tm, Topic t, Topic superclass)
boolean	runInOwnThread() Whether or not this tool should fork own thread.
boolean	useURLCrawler()

Methods inherited from class org.wandora.application.tools.extractors.AbstractExtractor

_extractTopicsFrom, _extractTopicsFrom, _extractTopicsFrom, acceptBrowserExtractRequest, addCrawlerUrl, browserExtractorConsumesPlainText, buildSI, buildSL, clearMasterSubject, createAssociation, createAssociation, createTopic, createTopic, createTopic, createTopic, createTopic, createTopic, createTopic, croppedFilename, croppedFilename, croppedUrlString, croppedUrlString, doBrowserExtract, dropExtract, dropExtract, dropExtract, execute, extractTopicsFrom, extractTopicsFrom, extractTopicsFrom, extractTopicsFrom, extractTopicsFromText, getBrowserExtractorName, getCrawlerMode, getExtractorType, getForceContent, getForceFiles, getForceUrls, getGUIText, getGUIText, getInterruptsHandled, getMasterSubject, getType, getWandora, handle, handleContent, handleCustomType, handleFiles, handleForcedContent, handleInterrupt, handleStringContent, handleUrls, initializeCustomType, instantDropHandle, makeSubclassOfWandoraClass, setData, setDisplayName, setForceContent, setForceFiles, setForceUrls, setMasterSubject, setMasterSubject, setTopicMap, setupCrawler, setWandora, takeNap, urlEncode, useTempTopicMap

Methods inherited from class org.wandora.application.tools.AbstractWandoraTool

addUndoMarker, addUndoMarker, allowMultipleInvocations, clearAllThreads, clearThreads, clearThreads, clearToolLock, clearToolLock, clearToolLocks, configure, execute, execute, forceStop, forceStop, getContext, getCurrentLogger, getDefaultLogger, getHistory, getLastLogger, getState, getThreads, getThreads, getToolMenuItem, getToolMenuItem, getTopicName, hlog, initialize, interruptAllThreads, interruptThreads, interruptThreads, isConfigurable, isRunning, isRunning, lockLog, log, log, log, log, requiresRefresh, run, setContext, setDefaultLogger, setLogTitle, setProgress, setProgressMax, setState, setToolLogger, singleLog, singleLog, singleLog, solveContextTopicMap, solveNameForTopicMap, writeOptions

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.wandora.application.WandoraTool

configure, execute, execute, getContext, getToolMenuItem, hlog, initialize, isConfigurable, isRunning, log, log, log, log, requiresRefresh, setContext, setToolLogger, writeOptions

Methods inherited from interface org.wandora.application.WandoraToolLogger

forceStop, getHistory, getState, lockLog, setLogTitle, setProgress, setProgressMax, setState

Methods inherited from interface java.lang.Runnable

run

Field Detail

contentTypes

private final java.lang.String[] contentTypes

DBPEDIA_RESOURCE_SI

public static final java.lang.String DBPEDIA_RESOURCE_SI

See Also:

Constant Field Values

BYLINE_SI

public static final java.lang.String BYLINE_SI

See Also:

Constant Field Values

ARTICLE_SI

public static final java.lang.String ARTICLE_SI

See Also:

Constant Field Values

BODY_SI

public static final java.lang.String BODY_SI

See Also:

Constant Field Values

TEXT_SI

public static final java.lang.String TEXT_SI

See Also:

Constant Field Values

ABSTRACT_SI

public static final java.lang.String ABSTRACT_SI

See Also:

Constant Field Values

LEAD_PARAGRAPH_SI

public static final java.lang.String LEAD_PARAGRAPH_SI

See Also:

Constant Field Values

DATE_SI

public static final java.lang.String DATE_SI

See Also:

Constant Field Values

LANG_SI

public static final java.lang.String LANG_SI

See Also:

Constant Field Values

FACET_SI

public static final java.lang.String FACET_SI

See Also:

Constant Field Values

KEYWORD_SI

public static final java.lang.String KEYWORD_SI

See Also:

Constant Field Values

DES_FACET_SI

public static final java.lang.String DES_FACET_SI

See Also:

Constant Field Values

GEO_FACET_SI

public static final java.lang.String GEO_FACET_SI

See Also:

Constant Field Values

ORG_FACET_SI

public static final java.lang.String ORG_FACET_SI

See Also:

Constant Field Values

PER_FACET_SI

public static final java.lang.String PER_FACET_SI

See Also:

Constant Field Values

SOURCE_FACET_SI

public static final java.lang.String SOURCE_FACET_SI

See Also:

Constant Field Values

CLASSIFIER_FACET_SI

public static final java.lang.String CLASSIFIER_FACET_SI

See Also:

Constant Field Values

COLUMN_FACET_SI

public static final java.lang.String COLUMN_FACET_SI

See Also:

Constant Field Values

MATERIAL_TYPE_FACET_SI

public static final java.lang.String MATERIAL_TYPE_FACET_SI

See Also:

Constant Field Values

NYT_SI

public static final java.lang.String NYT_SI

See Also:

Constant Field Values

EVENT_SI

public static final java.lang.String EVENT_SI

See Also:

Constant Field Values

EVENT_DETAIL_URL_SI

public static final java.lang.String EVENT_DETAIL_URL_SI

See Also:

Constant Field Values

EVENT_NAME_SI

public static final java.lang.String EVENT_NAME_SI

See Also:

Constant Field Values

EVENT_DESCRIPTION_SI

public static final java.lang.String EVENT_DESCRIPTION_SI

See Also:

Constant Field Values

EVENT_VENUE_SI

public static final java.lang.String EVENT_VENUE_SI

See Also:

Constant Field Values

EVENT_CATEGORY_SI

public static final java.lang.String EVENT_CATEGORY_SI

See Also:

Constant Field Values

EVENT_LATITUDE_SI

public static final java.lang.String EVENT_LATITUDE_SI

See Also:

Constant Field Values

EVENT_LONGITUDE_SI

public static final java.lang.String EVENT_LONGITUDE_SI

See Also:

Constant Field Values

EVENT_DATE_SI

public static final java.lang.String EVENT_DATE_SI

See Also:

Constant Field Values

START_DATE_SI

public static final java.lang.String START_DATE_SI

See Also:

Constant Field Values

END_DATE_SI

public static final java.lang.String END_DATE_SI

See Also:

Constant Field Values

WEEKDAY_SI

public static final java.lang.String WEEKDAY_SI

See Also:

Constant Field Values

RECURRING_DAY_SI

public static final java.lang.String RECURRING_DAY_SI

See Also:

Constant Field Values

abbrToDayOfWeek

static final java.util.Map<java.lang.String,AbstractNYTExtractor.DayOfWeek> abbrToDayOfWeek

Constructor Detail

AbstractNYTExtractor

public AbstractNYTExtractor()

Method Detail

getName

public java.lang.String getName()

Description copied from class: AbstractWandoraTool

Tools name represent the tool in UI unless the tool has been given explicitly another GUI name. All tools should override this method and return a valid string representing name of the tool.

Specified by:

getName in interface WandoraTool

Overrides:

getName in class AbstractExtractor

Returns:

String representing tool's name.

getDescription

public java.lang.String getDescription()

Description copied from class: AbstractWandoraTool

AdminToolManager views tool descriptions while user browses available tools and build user customizable GUI elements such as Tools menu. By default description equals the tool name. All tools should override this method.

Specified by:

getDescription in interface WandoraTool

Overrides:

getDescription in class AbstractExtractor

Returns:

String representing tool's description.

getIcon

public javax.swing.Icon getIcon()

Description copied from class: AbstractWandoraTool

All tools may have identifying graphic icon used within tool GUI elements. getIcon should return Icon object of the tool.

Specified by:

getIcon in interface WandoraTool

Overrides:

getIcon in class AbstractExtractor

Returns:

icon for the tool.

getContentTypes

public java.lang.String[] getContentTypes()

Description copied from interface: Handler

Returns an array of String containing the content-types this ContentHandler can process.

Specified by:

getContentTypes in interface Handler

Overrides:

getContentTypes in class AbstractExtractor

Returns:

String array representing content types.

useURLCrawler

public boolean useURLCrawler()

Overrides:

useURLCrawler in class AbstractExtractor

runInOwnThread

public boolean runInOwnThread()

Description copied from class: AbstractWandoraTool

Whether or not this tool should fork own thread. If own thread is allowed, the execution of the tool return immediately. If own thread is not allowed the thread entering initial execute method is used. Extending classes should override this method.

Overrides:

runInOwnThread in class AbstractExtractor

Returns:

true if the tool should should be ran in separate thread.

getLangTopic

protected static Topic getLangTopic(TopicMap tm)
                             throws TopicMapException

Throws:

TopicMapException

getBylineTopic

public static Topic getBylineTopic(java.lang.String byline,
                                   TopicMap tm)
                            throws TopicMapException

Throws:

TopicMapException

getBylineTypeTopic

public static Topic getBylineTypeTopic(TopicMap tm)
                                throws TopicMapException

Throws:

TopicMapException

getKeywordTopic

public static Topic getKeywordTopic(java.lang.String keyword,
                                    TopicMap tm)
                             throws TopicMapException

Throws:

TopicMapException

getKeywordNameTopic

public static Topic getKeywordNameTopic(java.lang.String name,
                                        TopicMap tm)
                                 throws TopicMapException

Throws:

TopicMapException

getKeywordTypeTopic

public static Topic getKeywordTypeTopic(TopicMap tm)
                                 throws TopicMapException

Throws:

TopicMapException

getDBpediaResourceTopic

public static Topic getDBpediaResourceTopic(java.lang.String res,
                                            TopicMap tm)
                                     throws TopicMapException

Throws:

TopicMapException

getDBpediaResourceTypeTopic

public static Topic getDBpediaResourceTypeTopic(TopicMap tm)
                                         throws TopicMapException

Throws:

TopicMapException

getAbstractTypeTopic

public static Topic getAbstractTypeTopic(TopicMap tm)
                                  throws TopicMapException

Throws:

TopicMapException

getBodyTypeTopic

public static Topic getBodyTypeTopic(TopicMap tm)
                              throws TopicMapException

Throws:

TopicMapException

getTextTypeTopic

public static Topic getTextTypeTopic(TopicMap tm)
                              throws TopicMapException

Throws:

TopicMapException

getLeadParagraphTypeTopic

public static Topic getLeadParagraphTypeTopic(TopicMap tm)
                                       throws TopicMapException

Throws:

TopicMapException

getArticleTypeTopic

public static Topic getArticleTypeTopic(TopicMap tm)
                                 throws TopicMapException

Throws:

TopicMapException

getDateTypeTopic

public static Topic getDateTypeTopic(TopicMap tm)
                              throws TopicMapException

Throws:

TopicMapException

getNYTTypeTopic

public static Topic getNYTTypeTopic(TopicMap tm)
                             throws TopicMapException

Throws:

TopicMapException

getWandoraClassTopic

protected static Topic getWandoraClassTopic(TopicMap tm)
                                     throws TopicMapException

Throws:

TopicMapException

getOrCreateTopic

protected static Topic getOrCreateTopic(TopicMap tm,
                                        java.lang.String si)
                                 throws TopicMapException

Throws:

TopicMapException

getOrCreateTopic

protected static Topic getOrCreateTopic(TopicMap tm,
                                        java.lang.String si,
                                        java.lang.String bn)
                                 throws TopicMapException

Throws:

TopicMapException

makeSubclassOf

protected static void makeSubclassOf(TopicMap tm,
                                     Topic t,
                                     Topic superclass)
                              throws TopicMapException

Throws:

TopicMapException

getEventTypeTopic

public static Topic getEventTypeTopic(TopicMap tm)
                               throws TopicMapException

Throws:

TopicMapException

getDescriptionTypeTopic

public static Topic getDescriptionTypeTopic(TopicMap tm)
                                     throws TopicMapException

Throws:

TopicMapException

getLatitudeTypeTopic

public static Topic getLatitudeTypeTopic(TopicMap tm)
                                  throws TopicMapException

Throws:

TopicMapException

getLongitudeTypeTopic

public static Topic getLongitudeTypeTopic(TopicMap tm)
                                   throws TopicMapException

Throws:

TopicMapException

getVenueTopic

public static Topic getVenueTopic(java.lang.String venue,
                                  TopicMap tm)
                           throws TopicMapException

Throws:

TopicMapException

getVenueTypeTopic

public static Topic getVenueTypeTopic(TopicMap tm)
                               throws TopicMapException

Throws:

TopicMapException

getCategoryTopic

public static Topic getCategoryTopic(java.lang.String category,
                                     TopicMap tm)
                              throws TopicMapException

Throws:

TopicMapException

getCategoryTypeTopic

public static Topic getCategoryTypeTopic(TopicMap tm)
                                  throws TopicMapException

Throws:

TopicMapException

getEventDateTypeTopic

public static Topic getEventDateTypeTopic(TopicMap tm)
                                   throws TopicMapException

Throws:

TopicMapException

getStartDateTypeTopic

public static Topic getStartDateTypeTopic(TopicMap tm)
                                   throws TopicMapException

Throws:

TopicMapException

getEndDateTypeTopic

public static Topic getEndDateTypeTopic(TopicMap tm)
                                 throws TopicMapException

Throws:

TopicMapException

getDayOfWeekTypeTopic

public static Topic getDayOfWeekTypeTopic(TopicMap tm)
                                   throws TopicMapException

Throws:

TopicMapException

getRecurringDayTypeTopic

public static Topic getRecurringDayTypeTopic(TopicMap tm)
                                      throws TopicMapException

Throws:

TopicMapException

getRecurringDayTopic

public static Topic getRecurringDayTopic(TopicMap tm,
                                         java.lang.String abbr)
                                  throws TopicMapException

Throws:

TopicMapException