abspython v1.0

In the previous post, we have configured our environment for packaging. Now we are going to package a small module named d3-time from D3.js .

First of all, what is d3-time? It's a replacement for the default JavaScript date-time module.

Make a directory of your choice, and cd to it

$ mkdir pac_d3
$ cd pac_d3

We need to install the d3-time module by using npm,

$ npm i d3-time

From here onwards is the real core part of packaging..

Now we are going to pull the latest d3-time using npm2deb(packaging tool for npm).

$ npm2deb create d3-time

Downloading source tarball file using debian/watch file...
uscan: Newest version of node-d3-time on remote site is 1.0.11, specified download version is 1.0.11
Successfully symlinked ../node-d3-time-1.0.11.tar.gz to ../node-d3-time_1.0.11.orig.tar.gz.
Creating debian source package...
 .....
internal/modules/cjs/loader.js:583
    throw err;
    ^
Error: Cannot find module '.'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:581:15)
    at Function.Module._load (internal/modules/cjs/loader.js:507:25)
    at Module.require (internal/modules/cjs/loader.js:637:17)
    at require (internal/modules/cjs/helpers.js:22:18)
    at [eval]:1:1
    at Script.runInThisContext (vm.js:96:20)
    at Object.runInThisContext (vm.js:303:38)
    at Object.<anonymous> ([eval]-wrapper:6:22)
    at Module._compile (internal/modules/cjs/loader.js:689:30)
    at evalScript (internal/bootstrap/node.js:587:27)
dh_auto_test: /usr/bin/node -e require\(\".\"\) returned exit code 1
make: *** [debian/rules:8: build] Error 255
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
 .....
Remember, your new source directory is d3-time/node-d3-time-1.0.11
This is not a crystal ball, so please take a look at auto-generated files.
You may want fix first these issues:
d3-time/node-d3-time-1.0.11/debian/control:Description: FIX_ME write the Debian package description
d3-time/node-d3-time_itp.mail:Subject: ITP: node-d3-time -- FIX_ME write the Debian package description
d3-time/node-d3-time_itp.mail:  Description     : FIX_ME write the Debian package description
d3-time/node-d3-time_itp.mail: FIX_ME: This ITP report is not ready for submission, until you are
d3-time/node-d3-time_itp.mail:FIX_ME: Explain why this package is suitable for adding to Debian. Is
d3-time/node-d3-time_itp.mail:FIX_ME: Explain how you intend to consistently maintain this package

We got a lot of errors from the first one itself. So we need to manually fix these errors.

After the above command, we have a new folder named d3-time. This the folder that is pulled from Debian watch file ie latest files(unstable) . The folder structure of d3-time is,

drwxr-xr-x 4 dev dev  4096 Aug 25 04:17 .
drwxr-xr-x 4 dev dev  4096 Aug 25 04:16 ..
drwxr-xr-x 3 dev dev  4096 Aug 25 04:16 node-d3-time
drwxr-xr-x 5 dev dev  4096 Aug 25 04:16 node-d3-time-1.0.11
-rw-r--r-- 1 dev dev 36662 Aug 25 04:16 node-d3-time-1.0.11.tar.gz
-rw-r--r-- 1 dev dev  2336 Aug 25 04:17 node-d3-time_1.0.11-1.debian.tar.xz
-rw-r--r-- 1 dev dev  1144 Aug 25 04:17 node-d3-time_1.0.11-1.dsc
lrwxrwxrwx 1 dev dev    26 Aug 25 04:16 node-d3-time_1.0.11.orig.tar.gz -> node-d3-time-1.0.11.tar.gz
-rw-r--r-- 1 dev dev  1574 Aug 25 04:16 node-d3-time_itp.mail

We are going to package the node-d3-time-1.0.11, which is the latest package. We first try to build the package by using dpkg-buildpackage inside the node-d3-time-1.0.11.

P.S: While building make sure you are outside the debian/ directory

 ....
internal/modules/cjs/loader.js:583
    throw err;
    ^
Error: Cannot find module '.'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:581:15)
    at Function.Module._load (internal/modules/cjs/loader.js:507:25)
    at Module.require (internal/modules/cjs/loader.js:637:17)
    at require (internal/modules/cjs/helpers.js:22:18)
    at [eval]:1:1
    at Script.runInThisContext (vm.js:96:20)
    at Object.runInThisContext (vm.js:303:38)
    at Object.<anonymous> ([eval]-wrapper:6:22)
    at Module._compile (internal/modules/cjs/loader.js:689:30)
    at evalScript (internal/bootstrap/node.js:587:27)
dh_auto_test: /usr/bin/node -e require\(\".\"\) returned exit code 1
make: *** [debian/rules:8: build] Error 255
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2

Oops. We got new errors. In every situation error is our friend. All error indicates our mistakes. Here, the tool is searching for module "." which is weird. Let us check the package.json file.

Look for a field main which tells nodejs to load a particular file when that directory is inside a require statement. For example when you have require('d3-time'); it looks for a directory called d3-time inside node_modules directory or in directories mentioned in NODE_PATH variable or default load path set by nodejs. In debian, /usr/share/nodejs is the current preferred path. Once it finds the matching directory, it looks for package.json file inside it and loads the file mentioned in main field. You can see "main": "dist/d3-time.js", But if you look at your local directory, this file is not present. Usually dist contains files generated from source by tools like rollup, webpack, babel etc

{
.......
	"pretest": "rollup -c",
.......
  },
  "devDependencies": {
    "eslint": "5",
    "rollup": "0.64",
    "rollup-plugin-terser": "1",
    "tape": "4"
  }
}

Here, the corresponding command in pretest is rollup -c (in most packages it is build instead of pretest) and rollup version required is 0.64. So, we need to make a file named build in debian/nodejs/ and add rollup -c. And try building again.

 ....
Found debian/nodejs/./build
        sh -e debian/nodejs/build
[!] Error: Unexpected token
rollup.config.js (22:4)
20:   config,
21:   {
22:     ...config,
        ^
23:     output: {
24:       ...config.output,
dh_auto_build: sh -e debian/nodejs/build returned exit code 1
make: *** [debian/rules:8: build] Error 255
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2

This is a syntax error in the rollup.config.js file. After some discussion with JS Developers, we understand that it is due to rollup not understand the syntax.

We can check the version of rollup installed by,

$ apt policy rollup

rollup:
  Installed: 0.50.0-6
  Candidate: 0.50.0-6
  Version table:
 *** 0.50.0-6 500
        500 http://deb.debian.org/debian sid/main amd64 Packages
        100 /var/lib/dpkg/status

We know that the d3-time require rollup of version 0.64 in the package.json file. The solution is strip down the rollup.config.js to,

import * as meta from "./package.json";

const config = {
  input: "src/index.js",
  external: Object.keys(meta.dependencies || {}).filter(key => /^d3-/.test(key)),
  output: {
    file: `dist/${meta.name}.js`,
    name: "d3",
    format: "umd",
    indent: false,
    extend: true,
    banner: `// ${meta.homepage} v${meta.version} Copyright ${(new Date).getFullYear()} ${meta.author.name}`,
    globals: Object.assign({}, ...Object.keys(meta.dependencies || {}).filter(key => /^d3-/.test(key)).map(key => ({[key]: "d3"})))
  },
  plugins: []
};

export default [
  config
];

And since these are upstream files, we are not allowed to directly edit the files (except debian/*). Thus, we need to apply a patch.

Patch is applied by,

$ dpkg-source --commit

Add a relevant file named similar to something-config.patch and add short description on it.

If you messed up the patch and would like to revert it do

$ quilt pop -a

And delete .pc and debian/patches and restart the patching.

After building again,

 ....
dpkg-deb: building package 'node-d3-time' in '../node-d3-time_1.0.11-2_all.deb'.
 dpkg-genbuildinfo
 dpkg-genchanges  >../node-d3-time_1.0.11-2_amd64.changes
dpkg-genchanges: info: including full source code in upload
 dpkg-source --after-build .
dpkg-buildpackage: info: full upload (original source is included)

We got the .deb in the parent directory. 📦

But this is not over quite. Remember we remove some code. Actually we want the same feature. Here, we are going to use uglifyjs.terser as replacement of the previously deleted code.

uglifyjs.terser dist/d3-time.js -o dist/d3-time.min.js

Append this code to the build file, i.e debian/nodejs/build, and build again.

P.S: Uglifyjs.terser is a fork of uglyifyjs and it is used to make the *.min.js files which are supposed to create by terser in rollup(but the rollup in debian at this time is outdated)

 ....
dpkg-source: info: building node-d3-time using existing ./node-d3-time_1.0.11.orig.tar.gz
dpkg-source: info: using patch list from debian/patches/series
dpkg-source: info: local changes detected, the modified files are:
 node-d3-time-1.0.11/dist/d3-time.js
dpkg-source: info: you can integrate the local changes with dpkg-source --commit
dpkg-source: error: aborting due to unexpected upstream changes, see /tmp/node-d3-time_1.0.11-2.diff.saoZeU
dpkg-buildpackage: error: dpkg-source -b . subprocess returned exit status 2

We need to add the two files generated while building to debian/clean.

dist/d3-time.js
dist/d3-time.min.js

Also we need to update the Build-depends in debian/control .

....

Build-Depends:
 debhelper-compat (= 11)
 , nodejs (>= 6)
 , pkg-js-tools (>= 0.8.10)
 , rollup
 , uglifyjs.terser
Standards-Version: 4.4.0
Homepage: https://d3js.org/d3-time/
Vcs-Git: https://salsa.debian.org/js-team/node-d3-time.git
Vcs-Browser: https://salsa.debian.org/js-team/node-d3-time

....

This should be it. Try building again.

 ....
dpkg-deb: building package 'node-d3-time' in '../node-d3-time_1.0.11-2_all.deb'.
 dpkg-genbuildinfo
 dpkg-genchanges  >../node-d3-time_1.0.11-2_amd64.changes
dpkg-genchanges: info: including full source code in upload
 dpkg-source --after-build .
dpkg-buildpackage: info: full upload (original source is included)

Yeah. It worked and successful made a .deb file in the parent directory.

Now, we need to check the .deb file have no error. Here, we use tool named lintian to check for errors in .deb file.

$ lintian ../node-d3-time_1.0.11-3_all.deb

N: Using profile debian/main.
N: Starting on group node-d3-time/1.0.11-4
N: Unpacking packages in group node-d3-time/1.0.11-4
N: ----
N: Processing binary package node-d3-time (version 1.0.11-4, arch all) ...
N: Finished processing group node-d3-time/1.0.11-4

For me, lintian didn't show any error. The deb file can be only sent to upstream if it has no errors.

Always remember to make linitian happy. 😀

In the next post, we are going to learn how to upgrade an existing package to its latest upstream version.

:wq for today.

Debian Packaging Part-2