Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: kanzure/papermonk-downloader-plosone
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 57f71eed7410
Choose a base ref
...
head repository: kanzure/papermonk-downloader-plosone
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 4363ee99d22a
Choose a head ref
  • 4 commits
  • 4 files changed
  • 1 contributor

Commits on Aug 16, 2013

  1. initial mocking of an http request 

    This is a simple mock that returns the expected html of the page to test
the downloader against. For now, the only tested property is the title
found in the html.
    @kanzure
    kanzure committed Aug 16, 2013
    Copy the full SHA
    6aec849 View commit details

  2. stop jsdom from hanging the entire test system 

    The window must be closed because otherwise the test suite never ends
itself.

Also, jsdom is really slow and is overkill for a task like "extract the
<title> of an html document". It's probably even slower inside a
browser's javascript environment, but I haven't tested that yet.
    @kanzure
    kanzure committed Aug 16, 2013
    Copy the full SHA
    cd0a295 View commit details

  3. Merge branch 'basic-http-mocking' into master

    @kanzure
    kanzure committed Aug 16, 2013
    Copy the full SHA
    1975e49 View commit details

  4. version bump to: v0.0.3 

    Hooray! HTTP mocking.
    @kanzure
    kanzure committed Aug 16, 2013
    Copy the full SHA
    4363ee9 View commit details
Showing with 2,243 additions and 1 deletion.
  1. +3 −0 index.js
  2. +2 −1 package.json
  3. +23 −0 tests.js
  4. +2,215 −0 tests/data/html/d6eabc5d85991d2248c6449d8be3d92b.html
3 changes: 3 additions & 0 deletions index.js
View file
Original file line number Diff line number Diff line change
@@ -66,6 +66,9 @@ module.exports.download = function download(url, options, callback) {
},
],
};

// otherwise node will hang
window.close();
}

callback(error, metadata);
3 changes: 2 additions & 1 deletion package.json
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "papermonk-downloader-plosone",
"description": "plosone.org scraper",
"version": "0.0.2",
"version": "0.0.3",
"readmeFilename": "README.md",
"homepage": "https://github.com/kanzure/papermonk-downloader-plosone",
"author": {
@@ -45,6 +45,7 @@
"url-parser": ">0",
"request": "~2.26.0",
"jsdom": "~0.8.0",
"nock": ">0.22.0",
"tape": ">0"
},
"optionalDependencies": {
23 changes: 23 additions & 0 deletions tests.js
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
var nock = require("nock");
var test = require("tape");

test("require against the module", function(t) {
@@ -51,3 +52,25 @@ test("matches for a url with plosone.org", function(t) {

t.end();
});

test("extracts the correct html title", function(t) {
// http://www.plosone.org/article/info:doi/10.1371/journal.pone.0071334
var downloader = require("./");

// mock data storage location
var filename = "d6eabc5d85991d2248c6449d8be3d92b"; // md5 of url
var filepath = __dirname + "/tests/data/html/" + filename + ".html";
var expected_title = "PLOS ONE: Expansion of Multipotent Stem Cells from the Adult Human Brain";

// setup the mock
var scope = nock("http://example.com")
.get("/paper")
.replyWithFile(200, filepath);

// make sure the title is extracted correctly
downloader.download("http://example.com/paper", {}, function(error, metadata) {
t.equal(expected_title, metadata.html.title, "title matches expectations");
t.ok(scope.isDone(), "nock scope is done");
t.end();
});
});
Loading