Feature request #4805

remove internal spatialite

Added by Brian Freed about 5 years ago. Updated over 3 years ago.

Status:Closed Start Date:01/13/2012
Priority:Normal Due date:
Assigned to:Jürgen Fischer % Done:

0%

Category:Data Provider
Target version:Future Release - Nice to have
Platform: Resolution:
Platform version: Pull Request or Patch supplied:No
Status info: Tag:

Description

Now that Sandro has officially released 3.0 stable, I was wondering when QGIS (specifically, the Windows pre-compiled executable) might include 3.0 instead of 2.4?

In 3.0, MakeLine() and MakePoint() are easier to use than LineFromText()/GeomFromText, and the RecoverSpatialIndex() function solves some issues I've noticed with QGIS and Spatialite tables with Spatial Indexes.

Actually, come to think of it, I think those issues themselves might go away - the function to make the Spatial Index update on object creation seems a little different than it was in 2.4, and I think that's the cause of objects created in QGIS not showing up visually in QGIS (despite showing up in the attribute editor) until I run RecoverSpatialIndex() on the table in Spatialite.

0001-drop-internal-spatialite-and-internal-spatialindex.patch.gz (1.8 MB) Jürgen Fischer, 04/17/2012 02:09 pm

Associated revisions

Revision c56491b11124f9ceefa9b6b3d501374755aac84c
Added by Jürgen Fischer almost 5 years ago

fix #4805:
- drop internal spatialite and internal spatialindex
- drop support for debian lenny (no system spatialindex/spatialite there)

History

Updated by Jürgen Fischer about 5 years ago

  • Category changed from Build / Install to Data Provider
  • Assigned to set to Sandro Furieri

Updated by Sandro Furieri about 5 years ago

Hi Brian,

you can consult the expected roadmap for SpatiaLite 3.0x
(this including any update for the QGIS data provider) on
the SpatiaLite's own ML. please see:
http://groups.google.com/group/spatialite-users/browse_thread/thread/4f206ceb6d934dc2#
(my answer to Volker Froelich on Tue, 10 Jan 2012 17:45:43)

very shortly said:
1) we need to definitively fix 3.0x before
2) updating the QGIS data provider will then follow immediately afterwards

bye Sandro

Updated by Brian Freed about 5 years ago

wonderful, thanks!

Updated by Giovanni Manghi almost 5 years ago

  • Target version changed from Version 1.7.4 to Version 2.0.0

Updated by Sandro Furieri almost 5 years ago

Short recall:
- QGIS has its own SpatiaLite data provider: this is intended
to be safely operable using any version of libspatialite
- and in turns libspatialite depends on few other libraries:
libsqlite, libproj, libgeos_c, libgeos, libiconv [libfreeexl]

mainly for historical reasons QGIS includes a "private internal
copy" of libspatialite-amalgamation (which in turn supports a
private internal copy of libsqlite).
few years ago this one seemed a reasonable solution to be
adopted, simply because libspatialite wasn't supported at all
on many platforms, and libsqlite too was unconsistently supported:
using private internal copies obviously resolved this issue.

anyway such solution isn't neither elegant nor safe (as confirmed
by direct field experience): QGIS is a very complex package with
ramified dependencies (this including Python). so a real risk
to deploy many different (and conflicting) libsqlite and libspatialite
versions exists, thus causing troubles and stability issues.

the obvious solution is to never use the internal copies of libsqlite
and libspatialite internally shipped by QGIS.
using the "system-wide" versions of both libraries is a much better
design choice, always ensuring consistent usage of the same dynamic
library version for any possible package/component.

nowadays both libspatialite/libsqlite are decently supported on many
Linux distros; and have recently been supported by OsGeo4W too.
so there is absolutely no good reason suggesting to still continue
using the doomed "internal copies": their usage should be considered
as highly deprecated and intrinsically unsafe.

accordingly to all the above considerations, I suppose that completely
removing any "internal copy" from QGIS (so effectively imposing to always
use system-wide dynamic libraries) should be a really good idea.

Updated by Jürgen Fischer almost 5 years ago

  • Assigned to changed from Sandro Furieri to Jürgen Fischer
  • Platform deleted (Windows)

Ok, but pyspatialite (dependency of dbmanager) is not that common. It's in OSGeo4W, but not in debian and ubuntu yet. For debian unstable it's close. So we should keep that for now.

The attached patch removes internal spatialite and internal spatialindex (which doesn't build on windows anyway) and also fixes the dbmanager build issue. As libspatialindex is used on Debian lenny, it also drops support for that.

But should we apply that to current master?

Updated by Jürgen Fischer almost 5 years ago

  • File 0001-drop-internal-spatialite-and-internal-spatialindex.patch.gz added

Updated by Jürgen Fischer almost 5 years ago

  • File deleted (0001-drop-internal-spatialite-and-internal-spatialindex.patch.gz)

Updated by Jürgen Fischer almost 5 years ago

Linux and Windows are fine with this. What's the situation on OS X?

Updated by Sandro Furieri almost 5 years ago

Hi Jürgen,

I fully agree with your considerations: pyspatialite seems to be
a key component, because QGIS strongly relies on Python modules.
Please note: I'm not the developer of pyspatialite: he's Lokkju
Brennr <>

It's really a pity that Debian-GIS doesn't support pyspatialite:
I'll try to ask Francesco P. Lovergine <> about
this; may well be this could resolve (hope well ...)

I've not yet tested the most recent version of pyspatialite; anyway,
if I correctly remember, the pyspatialite own build script was rather
problematic, and surely requiring substantial adjustments:
- it pretended to download libspatialite-amalgamation sources from the
download site before attempting to build
- it required the amalgamation (no attempt to locate external libsqlite3)
I imagine that only Lokkju could fix such issues.

Updated by Jürgen Fischer almost 5 years ago

Sandro Furieri wrote:

It's really a pity that Debian-GIS doesn't support pyspatialite: I'll try to ask Francesco P. Lovergine <> about this; may well be this could resolve (hope well ...)

David Paleino <> is already working on a package (in Debian-GIS)

I imagine that only Lokkju could fix such issues.

I didn't check the latest version either, but we have an internal pyspatialite too - that was what I meant to "keep for now".

Updated by Jürgen Fischer almost 5 years ago

Jürgen Fischer wrote:

David Paleino <> is already working on a package (in Debian-GIS)

See also pyspatialite_3.0.1-2_i386.changes ACCEPTED into unstable

Updated by Jürgen Fischer almost 5 years ago

  • Subject changed from Update QGIS' Spatialite to 3.0 to remove internal spatialite

Updated by William Kyngesburye almost 5 years ago

Jürgen Fischer wrote:

Linux and Windows are fine with this. What's the situation on OS X?

At least for my builds (and from source based on the INSTALL instructions), I don't use the internal spatialite. My SQLite framework includes spatialite. It also includes pyspatialite simply as an symlink to pysqlite.

Others who build from source with the package managers (fink, macports) probably have spatialite, maybe pyspatialite (at least pysqlite).

Updated by Sandro Furieri almost 5 years ago

following Williams's suggestions, I've just performed few tests
about the pysqlite connector; here are my findings.

from pysqlite2 import dbapi2 as db
con = db.connect(':memory:')
con.enable_load_extension(True)
con.load_extension('libspatialite.so')
con.execute('select InitSpatialMetadata()')
for row in con.execute('select * from spatial_ref_sys'):
    print row

the above code snippet nicely works on both Debian and Mint
(and I easily imagine on Ubuntu as well); accordingly to
William, the same seems to be for Mac Os X.
so it looks like there is absolutely no real reason suggesting
to use the specialized pyspatialite connector on these systems.

anyway I've got an unexpected failure on Fedora 16:
"enable_load_extension is not supported"
and I've got a second failure on Windows, using the Python
console shipped within the OsGeo4W supplied most recent QGIS
nightly build: "unknown package: pysqlite2"

support for the pysqlite2 connector apparently seems to be
rather whimsical and erratic, and not at all consistent
between different platforms ... a really unpleasant condition.

and a real pity too, because simply using pysqlite2 thus
completely avoiding to use pyspatialite will surely be
a real progress IMHO

Updated by Jürgen Fischer almost 5 years ago

  • Status changed from Open to Closed

Updated by Jürgen Fischer almost 5 years ago

  • Status changed from Closed to In Progress

Jürgen Fischer wrote:

Fixed in changeset c56491b11124f9ceefa9b6b3d501374755aac84c.

argl, how could I miss that system spatialite is too old on Debian squeeze, Ubuntu maverick, natty and oneiric (all 2.4.0~rc2; complains about missing spatial_ref_sys_init) and Ubuntu lucid (2.3.0; also complains about various macros beginning with GAIA_POINTZM).

So I probably need to re-add internal spatialite - unless anyone has a better idea.

Updated by Paolo Cavallini almost 5 years ago

I assume you mean reenabling it just for the distro who ship old SL, right?

Updated by Jürgen Fischer almost 5 years ago

Paolo Cavallini wrote:

I assume you mean reenabling it just for the distro who ship old SL, right?

right.

Updated by Pirmin Kalberer over 4 years ago

  • Target version changed from Version 2.0.0 to Future Release - Nice to have

Updated by Jürgen Fischer over 3 years ago

  • Status changed from In Progress to Closed

Also available in: Atom