开发者

Using Java (HTMLUnit) to Scrape a Web Page then Write Results to a mySQL Database

开发者 https://www.devze.com 2023-01-07 22:14 出处:网络
I\'ve been able to use Java and HTMLUnit to scrape a web page, however I\'m unsure how I can get this to write the resulting data to a MySQL database on a remote hosted server. I also need this to hap

I've been able to use Java and HTMLUnit to scrape a web page, however I'm unsure how I can get this to write the resulting data to a MySQL database on a remote hosted server. I also need this to happen at regular intervals (perhaps once a day) if possible, wi开发者_如何学运维thout having to manually run the program.

Is this possible and can I have any guidance, thorough or not, on how to do this?

Thanks!


Get your page's HTML into some String variable

String scrapedHtml = "";

Create a table in mysql with a text field

create table html_store (
     id int primary key autoincrement,
     content text not null
);

Use JDBC code to connect to the database and inset the data. Set the connectionURL and other parameters correctly

Connection conn = DriverManager.getConnection(connectionURL, user, password);
PreparedStatement pstmt = 
    conn.prepareStatement("insert into html_store (content) values (?)");
pstmt.setString(1, scrapedHtml); 
pstmt.executeUpdate();
pstmt.close();
conn.close();
0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号