HOWTO: reduce dependencies of OSS packages PDF Print E-mail
User Rating: / 0
PoorBest 
Written by Michael Felt   

Reduce Dependencies? - Am I misleading you?

What I am really talking about is reducing the so-called "dependency-hell" when installing an OSS package. Somethimes they cannot avoided - but sometimes they can!

Shared Libraries - both a boon and a burden

When an OSS package uses another package this introduces a dependency. This means that to build, or install that OSS package the software it depends on needs to be installed first as well. When that also has a dependency the looping through packages to find (and install) grows.

A library such as libc.so (on Linix) or libc.a (on AIX) is extremely handy to have because, basically, if there is any part that depends on the C language it will need a "helper" function or procedure from libc.

But when I am building a package and it has a dependency on something obscure, or older than the "latest" it can become quite difficult to also build (and package) and install another OSS project that depends on a later version. In an ideal world the OSS package still working with an old version will update their sources so that they work with the new versions of the dependencies. But it doesn't always work that way.

And sometimes, it is just a hassle to keep 20+ different "dependencies" with their dependencies all ordered and tested. The idea is that a dependency (say OpenSSL) can just be updated and all the OSS packages that depend on OpenSSL are updated without any additional work. Often, that works.

Bored with specifying a long list of dependencies

That is my other problem. I would like, ideally, to just have one file that has all it needs - within itself. Once upon a time - and I remember it fairly well (mid 1980's) the concept of shared libraries was discussed as a way to lower system resource requirements - both memory and disk. FYI: in those days a disk of 20 MByte was huge - and system memory > 128K byte was huge.

Yes - I know my applications might be smaller, and the total amount in system memory could also be smaller, but I am actually trying - as much as possible to use static linked libraries rather than shared libraries. And, from many of the comments I read, sometimes between the lines on the "ld" man pages - static-linked programs can (if not should) run faster than programs with MANY rtld (run-time loader) symbol resolutions.

Impact?

The packages I am rebuilding (e.g., httpd, php) are having fewer dependancies. I have already learned that the way apache apr and apr-util are designed - they do not work as static with apache httpd and shared modules. The modules (many many of them) need stuff from apr and/or apr-util that the httpd "main" did not need and these functions and procedures turn up missing. Building the modules with static links to the functions means that some of the symbols get loaded multiple times - and httpd core dumps. Sigh.

So, yes. There is an impact. But the impact I am looking for is software packages that are easier to maintain - for a "consumer" such as an AIX admin (like you?) even if it means more work in the evening getting all the pieces together.

Update: Seems some packages won't work with static libraries - because not everything they need is loaded - or more accurate - other packages need that "static" API as well, but there is overlap - and more. So while partX is fine, trying to use partY causes a core dump because the same routines get statically linked - and a bit more. Then the same routines are statically linked multiple times - and their static variables and routines are not the ones that get run 'all the time'.

My goal is not changed - but getting there is much harder than I had hoped. Sigh!

My goal

More stable packages and fewer (ideally none) for most OSS projects - rather than havig to download and install 4 or 5 other things before "what it is all about" can finally be installed.

 
< Prev   Next >
109 queries executed
1
SET sql_mode = 'MYSQL40'
2 SELECT folder, element, published, params FROM jos_mambots WHERE published >= 1 AND access <= 0 AND folder = 'system' ORDER BY ordering
3 SELECT template FROM jos_templates_menu WHERE client_id = 0 AND ( menuid = 0 OR menuid = 169 ) ORDER BY menuid DESC LIMIT 1
4 DELETE FROM jos_session WHERE ( ( time < '1548313776' ) AND guest = 0 AND gid > 0 ) OR ( ( time < '1548313776' ) AND guest = 1 AND userid = 0 )
5 SELECT COUNT(*) FROM jos_stats_agents WHERE agent = 'Unknown' AND type = 0
6 UPDATE jos_stats_agents SET hits = ( hits + 1 ) WHERE agent = 'Unknown' AND type = 0
7 SELECT COUNT(*) FROM jos_stats_agents WHERE agent = 'Unknown' AND type = 1
8 UPDATE jos_stats_agents SET hits = ( hits + 1 ) WHERE agent = 'Unknown' AND type = 1
9 SELECT COUNT(*) FROM jos_stats_agents WHERE agent = 'com' AND type = 2
10 UPDATE jos_stats_agents SET hits = ( hits + 1 ) WHERE agent = 'com' AND type = 2
11 SELECT * FROM jos_menu WHERE id = 169
12 SELECT a.*, u.name AS author, u.usertype, cc.name AS category, s.name AS section, g.name AS groups, s.published AS sec_pub, cc.published AS cat_pub, s.access AS sec_access, cc.access AS cat_access, s.id AS sec_id, cc.id as cat_id FROM jos_content AS a LEFT JOIN jos_categories AS cc ON cc.id = a.catid LEFT JOIN jos_sections AS s ON s.id = cc.section AND s.scope = 'content' LEFT JOIN jos_users AS u ON u.id = a.created_by LEFT JOIN jos_groups AS g ON a.access = g.id WHERE a.id = 821 AND ( a.state = 1 OR a.state = -1 ) AND ( a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2019-01-24 08:24' ) AND ( a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2019-01-24 08:24' ) AND a.access <= 0
13 SELECT a.id FROM jos_content AS a WHERE a.catid = 74 AND a.state = 1 AND a.access <= 0 AND ( a.state = 1 OR a.state = -1 ) AND ( a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2019-01-24 08:24' ) AND ( a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2019-01-24 08:24' ) ORDER BY a.ordering
14 SELECT ROUND( v.rating_sum / v.rating_count ) AS rating, v.rating_count FROM jos_content AS a LEFT JOIN jos_content_rating AS v ON a.id = v.content_id WHERE a.id = 821
15 UPDATE jos_content SET hits = ( hits + 1 ) WHERE id = '821'
16 SELECT hits FROM jos_core_log_items WHERE time_stamp = '2019-01-24' AND item_table = '#__content' AND item_id = '821'
17 UPDATE jos_core_log_items SET hits = ( hits + 1 ) WHERE time_stamp = '2019-01-24' AND item_table = '#__content' AND item_id = '821'
18 SELECT folder, element, published, params FROM jos_mambots WHERE access <= 0 AND folder = 'content' ORDER BY ordering
19 SELECT value FROM jos_bookmarks_prefs WHERE userid = '-1' AND category = 'params' AND name = 'urlkey'
20 SELECT value FROM jos_bookmarks_prefs WHERE userid = '-1' AND category = 'params' AND name = 'snapshotactiv'
21 SELECT value FROM jos_bookmarks_prefs WHERE userid = '-1' AND category = 'params' AND name = 'snapshotother'
22 SELECT id FROM jos_mambots WHERE element = 'mosbookmarks' AND folder = 'content'
23 SELECT * FROM jos_mambots WHERE id = '32'
24 SELECT name, value FROM jos_bookmarks_prefs WHERE userid = '-1' AND category = 'params' ORDER BY name
25 SELECT c.id, c.parent FROM jos_bookmarks_categories as c WHERE c.id = -1
26 SELECT * FROM jos_bookmarks_columns WHERE category='' AND custom='0'
27 SELECT * FROM jos_bookmarks_columns WHERE category='' ORDER BY ordering
28 SELECT * FROM jos_bookmarks_columns WHERE category='admin'
29 SELECT * FROM jos_bookmarks_columns WHERE category='admin' ORDER BY ordering
30 SELECT id FROM jos_mambots WHERE element = 'plugin_jw_allvideos' AND folder = 'content'
31 SELECT * FROM jos_mambots WHERE id = '27'
32 SELECT id FROM jos_mambots WHERE element = 'plugin_jw_allvideos' AND folder = 'content'
33 SELECT * FROM jos_mambots WHERE id = '27'
34 SELECT id FROM jos_mambots WHERE element = 'plugin_jw_allvideos' AND folder = 'content'
35 SELECT * FROM jos_mambots WHERE id = '27'
36 SELECT id FROM jos_mambots WHERE element = 'plugin_jw_allvideos' AND folder = 'content'
37 SELECT * FROM jos_mambots WHERE id = '27'
38 SELECT a.* FROM jos_components AS a WHERE ( a.admin_menu_link = 'option=com_syndicate' OR a.admin_menu_link = 'option=com_syndicate&hidemainmenu=1' ) AND a.option = 'com_syndicate'
39 SELECT m.id FROM jos_modules AS m WHERE m.module = 'mod_rssfeed' AND m.published = 1
40 SELECT id, title, module, position, content, showtitle, params FROM jos_modules AS m INNER JOIN jos_modules_menu AS mm ON mm.moduleid = m.id WHERE m.published = 1 AND m.access <= 0 AND m.client_id != 1 AND ( mm.menuid = 0 OR mm.menuid = 169 ) ORDER BY ordering
41 SELECT a.id, a.introtext, a.fulltext , a.images, a.attribs, a.title, a.state FROM jos_content AS a INNER JOIN jos_categories AS cc ON cc.id = a.catid INNER JOIN jos_sections AS s ON s.id = a.sectionid WHERE a.state = 1 AND a.access <= 0 AND cc.access <= 0 AND s.access <= 0 AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2019-01-24 08:24' ) AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2019-01-24 08:24' ) AND a.catid = 44 AND cc.published = 1 AND s.published = 1 ORDER BY a.ordering LIMIT 15
42 SELECT id FROM jos_mambots WHERE element = 'mosbookmarks' AND folder = 'content'
43 SELECT * FROM jos_mambots WHERE id = '32'
44 SELECT name, value FROM jos_bookmarks_prefs WHERE userid = '-1' AND category = 'params' ORDER BY name
45 SELECT c.id, c.parent FROM jos_bookmarks_categories as c WHERE c.id = -1
46 SELECT * FROM jos_bookmarks_columns WHERE category='' AND custom='0'
47 SELECT * FROM jos_bookmarks_columns WHERE category='' ORDER BY ordering
48 SELECT * FROM jos_bookmarks_columns WHERE category='admin'
49 SELECT * FROM jos_bookmarks_columns WHERE category='admin' ORDER BY ordering
50 SELECT id FROM jos_mambots WHERE element = 'plugin_jw_allvideos' AND folder = 'content'
51 SELECT * FROM jos_mambots WHERE id = '27'
52 SELECT id FROM jos_mambots WHERE element = 'plugin_jw_allvideos' AND folder = 'content'
53 SELECT * FROM jos_mambots WHERE id = '27'
54 SELECT id FROM jos_mambots WHERE element = 'plugin_jw_allvideos' AND folder = 'content'
55 SELECT * FROM jos_mambots WHERE id = '27'
56 SELECT id FROM jos_mambots WHERE element = 'plugin_jw_allvideos' AND folder = 'content'
57 SELECT * FROM jos_mambots WHERE id = '27'
58 SELECT id FROM jos_menu WHERE link = 'index.php?option=com_search' AND published = 1
59 SELECT m.*, sum(case when p.published=1 then 1 else 0 end) as cnt FROM jos_menu AS m LEFT JOIN jos_menu AS p ON p.parent = m.id WHERE m.menutype='rt_splitSuckerFish' AND m.published='1' AND m.access <= '0' GROUP BY m.id ORDER BY m.parent, m.ordering
60 SELECT m.* FROM jos_menu AS m WHERE menutype='rt_splitSuckerFish' AND m.published='1'
61 SELECT * FROM jos_banner WHERE showBanner=1
62 SELECT id FROM jos_mambots WHERE element = 'mosbookmarks' AND folder = 'content'
63 SELECT * FROM jos_mambots WHERE id = '32'
64 SELECT name, value FROM jos_bookmarks_prefs WHERE userid = '-1' AND category = 'params' ORDER BY name
65 SELECT c.id, c.parent FROM jos_bookmarks_categories as c WHERE c.id = -1
66 SELECT * FROM jos_bookmarks_columns WHERE category='' AND custom='0'
67 SELECT * FROM jos_bookmarks_columns WHERE category='' ORDER BY ordering
68 SELECT * FROM jos_bookmarks_columns WHERE category='admin'
69 SELECT * FROM jos_bookmarks_columns WHERE category='admin' ORDER BY ordering
70 SELECT id FROM jos_mambots WHERE element = 'plugin_jw_allvideos' AND folder = 'content'
71 SELECT * FROM jos_mambots WHERE id = '27'
72 SELECT id FROM jos_mambots WHERE element = 'plugin_jw_allvideos' AND folder = 'content'
73 SELECT * FROM jos_mambots WHERE id = '27'
74 SELECT id FROM jos_mambots WHERE element = 'plugin_jw_allvideos' AND folder = 'content'
75 SELECT * FROM jos_mambots WHERE id = '27'
76 SELECT id FROM jos_mambots WHERE element = 'plugin_jw_allvideos' AND folder = 'content'
77 SELECT * FROM jos_mambots WHERE id = '27'
78 SELECT name, value FROM jos_bookmarks_prefs WHERE userid = '-1' AND category = 'params' ORDER BY name
79 SELECT c.id, c.parent FROM jos_bookmarks_categories as c WHERE c.id = -1
80 SELECT * FROM jos_bookmarks_columns WHERE category='' AND custom='0'
81 SELECT * FROM jos_bookmarks_columns WHERE category='' ORDER BY ordering
82 SELECT * FROM jos_bookmarks_columns WHERE category='admin'
83 SELECT * FROM jos_bookmarks_columns WHERE category='admin' ORDER BY ordering
84 SELECT DISTINCT b.*, u.name AS username FROM jos_bookmarks_itemcat AS i LEFT JOIN jos_bookmarks AS b ON b.id = i.itemid LEFT JOIN jos_bookmarks_categories AS bc ON bc.id = i.catid LEFT JOIN jos_users AS u ON u.id = b.created_by WHERE ((b.published=1 AND b.approved=1) OR b.id IS NULL) AND i.catid IN (4) AND b.access <= '0' AND bc.published = '1' AND bc.access <= '0' AND ((b.private = 0) OR (b.private = 1 AND b.created_by = '0')) ORDER BY b.title ASC
85 SELECT bc.name as category FROM jos_bookmarks_itemcat i LEFT OUTER JOIN jos_bookmarks_categories bc ON (i.catid = bc.id) WHERE (i.itemid = 298) ORDER BY bc.name ASC
86 SELECT bc.name as category FROM jos_bookmarks_itemcat i LEFT OUTER JOIN jos_bookmarks_categories bc ON (i.catid = bc.id) WHERE (i.itemid = 265) ORDER BY bc.name ASC
87 SELECT bc.name as category FROM jos_bookmarks_itemcat i LEFT OUTER JOIN jos_bookmarks_categories bc ON (i.catid = bc.id) WHERE (i.itemid = 297) ORDER BY bc.name ASC
88 SELECT bc.name as category FROM jos_bookmarks_itemcat i LEFT OUTER JOIN jos_bookmarks_categories bc ON (i.catid = bc.id) WHERE (i.itemid = 296) ORDER BY bc.name ASC
89 SELECT bc.name as category FROM jos_bookmarks_itemcat i LEFT OUTER JOIN jos_bookmarks_categories bc ON (i.catid = bc.id) WHERE (i.itemid = 248) ORDER BY bc.name ASC
90 SELECT bc.name as category FROM jos_bookmarks_itemcat i LEFT OUTER JOIN jos_bookmarks_categories bc ON (i.catid = bc.id) WHERE (i.itemid = 246) ORDER BY bc.name ASC
91 SELECT name, value FROM jos_bookmarks_prefs WHERE userid = '-1' AND category = 'params' ORDER BY name
92 SELECT c.id, c.parent FROM jos_bookmarks_categories as c WHERE c.id = -1
93 SELECT * FROM jos_bookmarks_columns WHERE category='' AND custom='0'
94 SELECT * FROM jos_bookmarks_columns WHERE category='' ORDER BY ordering
95 SELECT * FROM jos_bookmarks_columns WHERE category='admin'
96 SELECT * FROM jos_bookmarks_columns WHERE category='admin' ORDER BY ordering
97 SELECT DISTINCT b.*, u.name AS username FROM jos_bookmarks_itemcat AS i LEFT JOIN jos_bookmarks AS b ON b.id = i.itemid LEFT JOIN jos_bookmarks_categories AS bc ON bc.id = i.catid LEFT JOIN jos_users AS u ON u.id = b.created_by WHERE ((b.published=1 AND b.approved=1) OR b.id IS NULL) AND i.catid IN (34) AND b.access <= '0' AND bc.published = '1' AND bc.access <= '0' AND ((b.private = 0) OR (b.private = 1 AND b.created_by = '0')) ORDER BY b.title ASC LIMIT 6
98 SELECT bc.name as category FROM jos_bookmarks_itemcat i LEFT OUTER JOIN jos_bookmarks_categories bc ON (i.catid = bc.id) WHERE (i.itemid = 142) ORDER BY bc.name ASC
99 SELECT bc.name as category FROM jos_bookmarks_itemcat i LEFT OUTER JOIN jos_bookmarks_categories bc ON (i.catid = bc.id) WHERE (i.itemid = 289) ORDER BY bc.name ASC
100 SELECT bc.name as category FROM jos_bookmarks_itemcat i LEFT OUTER JOIN jos_bookmarks_categories bc ON (i.catid = bc.id) WHERE (i.itemid = 242) ORDER BY bc.name ASC
101 SELECT bc.name as category FROM jos_bookmarks_itemcat i LEFT OUTER JOIN jos_bookmarks_categories bc ON (i.catid = bc.id) WHERE (i.itemid = 294) ORDER BY bc.name ASC
102 SELECT bc.name as category FROM jos_bookmarks_itemcat i LEFT OUTER JOIN jos_bookmarks_categories bc ON (i.catid = bc.id) WHERE (i.itemid = 284) ORDER BY bc.name ASC
103 SELECT bc.name as category FROM jos_bookmarks_itemcat i LEFT OUTER JOIN jos_bookmarks_categories bc ON (i.catid = bc.id) WHERE (i.itemid = 293) ORDER BY bc.name ASC
104 SELECT a.id, a.title, a.sectionid, a.catid, a.created, a.hits FROM jos_content AS a LEFT JOIN jos_content_frontpage AS f ON f.content_id = a.id INNER JOIN jos_categories AS cc ON cc.id = a.catid INNER JOIN jos_sections AS s ON s.id = a.sectionid WHERE ( a.state = 1 AND a.sectionid > 0 ) AND ( a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2019-01-24 08:24' ) AND ( a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2019-01-24 08:24' ) AND ((TO_DAYS('2019-01-24') - TO_DAYS(a.created)) <= '60') AND a.access <= 0 AND cc.access <= 0 AND s.access <= 0 AND s.published = 1 AND cc.published = 1 ORDER BY a.hits DESC LIMIT 3
105 SELECT m.* FROM jos_menu AS m WHERE menutype = 'Products' AND published = 1 AND access <= 0 ORDER BY parent, ordering
106 SELECT a.id, a.title, a.sectionid, a.catid FROM jos_content AS a LEFT JOIN jos_content_frontpage AS f ON f.content_id = a.id INNER JOIN jos_categories AS cc ON cc.id = a.catid INNER JOIN jos_sections AS s ON s.id = a.sectionid WHERE ( a.state = 1 AND a.sectionid > 0 ) AND ( a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2019-01-24 08:24' ) AND ( a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2019-01-24 08:24' ) AND a.access <= 0 AND cc.access <= 0 AND s.access <= 0 AND s.published = 1 AND cc.published = 1 ORDER BY a.hits DESC LIMIT 3
107 SELECT ms.id AS sid, ms.type AS stype, mc.id AS cid, mc.type AS ctype, i.id as sectionid, i.id As catid, ms.published AS spub, mc.published AS cpub FROM jos_content AS i LEFT JOIN jos_sections AS s ON i.sectionid = s.id LEFT JOIN jos_menu AS ms ON ms.componentid = s.id LEFT JOIN jos_categories AS c ON i.catid = c.id LEFT JOIN jos_menu AS mc ON mc.componentid = c.id WHERE ( ms.type IN ( 'content_section', 'content_blog_section' ) OR mc.type IN ( 'content_blog_category', 'content_category' ) ) AND i.id = 174 ORDER BY ms.type DESC, mc.type DESC, ms.id, mc.id
108 SELECT ms.id AS sid, ms.type AS stype, mc.id AS cid, mc.type AS ctype, i.id as sectionid, i.id As catid, ms.published AS spub, mc.published AS cpub FROM jos_content AS i LEFT JOIN jos_sections AS s ON i.sectionid = s.id LEFT JOIN jos_menu AS ms ON ms.componentid = s.id LEFT JOIN jos_categories AS c ON i.catid = c.id LEFT JOIN jos_menu AS mc ON mc.componentid = c.id WHERE ( ms.type IN ( 'content_section', 'content_blog_section' ) OR mc.type IN ( 'content_blog_category', 'content_category' ) ) AND i.id = 281 ORDER BY ms.type DESC, mc.type DESC, ms.id, mc.id
109 SELECT ms.id AS sid, ms.type AS stype, mc.id AS cid, mc.type AS ctype, i.id as sectionid, i.id As catid, ms.published AS spub, mc.published AS cpub FROM jos_content AS i LEFT JOIN jos_sections AS s ON i.sectionid = s.id LEFT JOIN jos_menu AS ms ON ms.componentid = s.id LEFT JOIN jos_categories AS c ON i.catid = c.id LEFT JOIN jos_menu AS mc ON mc.componentid = c.id WHERE ( ms.type IN ( 'content_section', 'content_blog_section' ) OR mc.type IN ( 'content_blog_category', 'content_category' ) ) AND i.id = 431 ORDER BY ms.type DESC, mc.type DESC, ms.id, mc.id