HTMLParser模块详解
使用HTMLParser模块解析HTML页面
HTMLParser是python用来解析html和xhtml文件格式的模块。它可以分析出html里面的标签、数据等等,是一种处理html的简便途径。HTMLParser采用的是一种事件驱动的模式,当HTMLParser找到一个特定的标记时,它会去调用一个用户定义的函数,以此来通知程序处理。它主要的回调函数的命名都是以handler_开头的,都HTMLParser的成员函数。当我们使用时,就从HTMLParser派生出新的类,然后重新定义这几个以handler_开头的函数即可。和在htmllib中的解析器不同,这个解析器并不是基于sgmllib模块的SGML解析器。
htmllib模块和sgmllib模块从Python2.6开始不鼓励使用,3.0以后的版本中被移除~~~
HTMLParser
class HTMLParser.HTMLParser
The HTMLParser class is instantiated without arguments.
HTMLParser类不需要参数进行实例化。
An HTMLParser instance is fed HTML data and calls handler functions when tags begin and end. The HTMLParser class is meant to be overridden by the user to provide a desired behavior.
一个HTMLParser实例传入HTML数据并且当传入开始和结束的tags参数时调用handler函数。HTMLParser类通过被用户重写方法来提供所需要的行为。
Unlike the parser in htmllib, this parser does not check that end tags match start tags or call the end-tag handler for elements which are closed implicitly by closing an outer element.
与htmllib中的解析器不同的是,这个解析器不检查结尾标签与开始标签的匹配或者对由关闭外部元素表明是关闭的元素调用结束标签handler
An exception is defined as well:
异常也被定义了:
exception HTMLParser.HTMLParseError
Exception raised by the HTMLParser class when it encounters an error while parsing. This exception provides three attributes: msg is a brief message explaining the error, lineno is the number of the line on which the broken construct was detected, and offset is the number of characters into the line at which the construct starts.
当遇到解析遇到错误时,该类将产生一个异常。该异常提供了三个属性:msg是用来解释错误的消息,lineno是检测到打断构造的行数,offset则是该行产生该构造的字符数。
HTMLParser instances have the following methods:
HTMLParser实例有以下方法:
HTMLParser.reset()
Reset the instance. Loses all unprocessed data. This is called implicitly at instantiation time.
重置该实例。失去所有未处理的数据。这个在实例化对象时被隐含地调用。
HTMLParser.feed(data)
Feed some text to the parser. It is processed insofar as it consists of complete elements; incomplete data is buffered until more data is fed or close() is called.
提供一些文本给解析器。在由完整元素组成的限度内进行处理,不完整的数据被缓冲直到更多的数据提供或者close()被调用。
HTMLParser.close()
Force processing of all buffered data as if it were followed by an end-of-file mark. This method may be redefined by a derived class to define additional processing at the end of the input, but the redefined version should always call theHTMLParser base class method close().
强制将所有的缓冲数据按跟在结束标记的数据一样进行处理。该方法可以通过派生类定义对输入结尾的额外处理来进行重定义,但是重定义的版本应该总是调用HTMLParser基类方法close()
HTMLParser.getpos()
Return current line number and offset.
返回当前行数和位移值。
HTMLParser.get_starttag_text()
Return the text of the most recently opened start tag. This should not normally be needed for structured processing, but may be useful in dealing with HTML “as deployed” or for re-generating input with minimal changes (whitespace between attributes can be preserved, etc.).
返回文本最近的开放标签。
HTMLParser.handle_starttag(tag, attrs)
This method is called to handle the start of a tag. It is intended to be overridden by a derived class; the base class implementation does nothing.
The tag argument is the name of the tag converted to lower case. The attrs argument is a list of (name, value) pairs containing the attributes found inside the tag’s <> brackets. The name will be translated to lower case, and quotes in the value have been removed, and character and entity references have been replaced. For instance, for the tag <A HREF="http://www.cwi.nl/">, this method would be called as handle_starttag('a', [('href', 'http://www.cwi.nl/')]).
该方法用来处理开始标签。其目的是被派生类重写;基类什么也不实现。tag参数是转换成小写的标签名称。attrs参数是一个(name,value)对包含了在标签<>中得属性。name将会转换成小写,并且value中得引号会被引出,并且字符串和实体引用将会被替代。
例如,对于标签<A HREF="http://www.cwi.nl/">,该方法将会调用为handle_starttag('a', [('href', 'http://www.cwi.nl/')])
Changed in version 2.6: All entity references from htmlentitydefs are now replaced in the attribute values.
2.6的版本变动:所有来自htmlentitydefs的实体引用现在在属性值中被替代了。
HTMLParser.handle_startendtag(tag, attrs)
Similar to handle_starttag(), but called when the parser encounters an XHTML-style empty tag (<a .../>). This method may be overridden by subclasses which require this particular lexical information; the default implementation 易做图 calls handle_starttag() and handle_endtag().
类似于handle_starttag(),不过用来处理遇到XHTML风格的空标签(<a .../>)。
HTMLParser.handle_endtag(tag)
This method is called to handle the end tag of an element. It is intended to be overridden by a derived class; the base class implementation does nothing. The tag argument is the name of the tag converted to lower case.
该方法用来处理元素的结束标签。
HTMLParser.handle_data(data)
This method is called to process arbitrary data. It is intended to be overridden by a derived class; the base class implementation does nothing.
该方法用来处理任意的数据。
HTMLParser.handle_charref(name)
This method is called to process a character reference of the form ref;. It is intended to be overridden by a derived class; the base class implementation does nothing.
该方法用来处理ref;形式的字符引用。
HTMLParser.handle_entityref(name)
This method is called to process a general entity reference of the form &name; where name is an general entity reference. It is intended to be overridden by a derived class; the base class implementation does nothing.
该方法用来处理形式&name;的一般实体引用,参数name是一般的实体引用。
HTMLParser.handle_comment(data)
This method is called when a comment is encountered. The comment argument is a string containing the text between the --and -- delimiters, but not the delimiters themselves. For example, the comment <!--text--> will cause this method to be called with the argument 'text'. It is intended to be overridden by a derived class; the base class implementation does nothing.
该方法用来处理遇到的评论。
HTMLParser.handle_decl(decl)
Method called when an SGML doctype declaration is read by the parser. The decl parameter will be the entire contents of the declaration inside the <!...> markup. It is intended to be overridden by a derived class; the base class implementation does nothing.
当解析器读到一个SGML的doctype声明该方法被调用。
HTMLParser.unknown_decl(data)
Method called when an unrecognized SGML declaration is read by the parser. The data parameter will be the entire contents of the declaration inside the <!...> markup. It is sometimes useful to be overridden by a derived class; the base class implementation throws an HTMLParseError.
当解析器读到一个未被识别SGML声明时将调用该方法。
HTMLParser.handle_pi(data)
Method called when a processing instruction is encountered. The data parameter will contain the entire processing instruction. For example, for the processing instruction <?proc color='red'>, this method would be called as handle_pi("proccolor='red'"). It is intended to be overridden
补充:web前端 , HTML 5 ,