weird image artifacts with JAI deferred loading
OpenGeo internal development for internal MassGIS enterprise contract.
Back to topic listing
Previous thread
-
From: groldan
Subject: weird image artifacts with JAI deferred loading
Date: Oct 27, 2009 08:59 AM
Hey, as you might know, the ArcSDE raster support is now based on JAI, like in using a custom ImageInputStream to read from a SDE raster query stream. For background: this ImageInputStream backs a RawImageInputStream. So the resulting RenderedImage for an ArcSDE coverage request is a small JAI ops chain like read->crop->scale->translate->mosaic (forget about the mosaic part when the arcsde raster is not a raster catalog -i.e., a runtime generated mosaic-) Everything works well as long as deferred loading of tiles is not used. To force that, I must call RenderedImage.getData() before returning the image resulting from the read operation. That is a tiled one matching the ArcSDE raster tile dimension. But if I don´t do so, strange artifacts appear in the WMS GetMap responses. Some seem to be avoided if I forcedly disable MetaTiling, like in commenting out the following code at GetMapReponse line 165: //if (MetatileMapProducer.isRequestTiled(request, delegate)) {... but others persist. Artifacts are either tiles with repeating patterns (like in a single tile being repeated) or images with blank rectangles. I suspected of the ESRI api's thread safety wrt internal tile buffers or something, but that might happen even for non tiling requests. So, (and apologies for the long preface), question is, does someone already experienced this kind of problems with JAI and deferred loading? Any known work around or thing to have in mind to get it working properly? I know it's hard to answer with such a vague description but if you ever experienced something similar please let me know. Cheers, Gabriel -- Gabriel Roldan OpenGeo - http://opengeo.org Expert service straight from the developers. -
From: groldan
Subject: looks like a jpeg bug? Re: [Geoserver-devel] weird image artifacts with JAI deferred loading
Date: Oct 27, 2009 09:43 AM
Hey, wrt the repeated tiles when metatiling is enabled, I found it only happens if the output format is JPEG, PNG and GIS are ok. The attached images show how tiling requests look when metatiling is activated or not, and output format is jpeg. any idea? Cheers, Gabriel Gabriel Roldan wrote: > Hey, > > as you might know, the ArcSDE raster support is now based on JAI, like > in using a custom ImageInputStream to read from a SDE raster query > stream. For background: this ImageInputStream backs a RawImageInputStream. > > So the resulting RenderedImage for an ArcSDE coverage request is a small > JAI ops chain like read->crop->scale->translate->mosaic (forget about > the mosaic part when the arcsde raster is not a raster catalog -i.e., a > runtime generated mosaic-) > > Everything works well as long as deferred loading of tiles is not used. > To force that, I must call RenderedImage.getData() before returning the > image resulting from the read operation. That is a tiled one matching > the ArcSDE raster tile dimension. > > But if I don´t do so, strange artifacts appear in the WMS GetMap > responses. Some seem to be avoided if I forcedly disable MetaTiling, > like in commenting out the following code at GetMapReponse line 165: > //if (MetatileMapProducer.isRequestTiled(request, delegate)) {... > but others persist. > Artifacts are either tiles with repeating patterns (like in a single > tile being repeated) or images with blank rectangles. > > I suspected of the ESRI api's thread safety wrt internal tile buffers or > something, but that might happen even for non tiling requests. > > So, (and apologies for the long preface), question is, does someone > already experienced this kind of problems with JAI and deferred loading? > Any known work around or thing to have in mind to get it working properly? > > I know it's hard to answer with such a vague description but if you ever > experienced something similar please let me know. > > Cheers, > Gabriel > -- Gabriel Roldan OpenGeo - http://opengeo.org Expert service straight from the developers.-
From: groldan
Subject: Re: looks like a jpeg bug? Re: [Geoserver-devel] weird image artifacts with JAI deferred loading
Date: Oct 27, 2009 09:45 AM
> > wrt the repeated tiles when metatiling is enabled, I found it only > happens if the output format is JPEG, PNG >and GIS are ok. I mean GIF
-
From: groldan
Subject: Re: [Geoserver-devel] looks like a jpeg bug? Re: weird image artifacts with JAI deferred loading
Date: Oct 28, 2009 05:18 PM
Hey Arne, that´s good to know... so it´s not me getting crazy, but a known issue. May be Simone or Daniele know something about that? Cheers, Gabriel Arne Kepp wrote: > I've had this same problem in GWC[1], it's been around for 2 years ++, > never understood how it could work in GeoServer, but then again I'm not > sure I ever configured a tiled=true client correctly either. (In my code > useJAI is set to false if it's JPEG) > > -Arne > > 1: > http://geowebcache.org/trac/browser/trunk/geowebcache/src/main/java/org/geowebcache/layer/wms/WMSMetaTile.java#L223 > > > > Gabriel Roldan wrote: >> Hey, >> >> wrt the repeated tiles when metatiling is enabled, I found it only >> happens if the output format is JPEG, PNG and GIS are ok. >> >> The attached images show how tiling requests look when metatiling is >> activated or not, and output format is jpeg. >> >> any idea? >> >> Cheers, >> Gabriel >> Gabriel Roldan wrote: >>> Hey, >>> >>> as you might know, the ArcSDE raster support is now based on JAI, >>> like in using a custom ImageInputStream to read from a SDE raster >>> query stream. For background: this ImageInputStream backs a >>> RawImageInputStream. >>> >>> So the resulting RenderedImage for an ArcSDE coverage request is a >>> small JAI ops chain like read->crop->scale->translate->mosaic (forget >>> about the mosaic part when the arcsde raster is not a raster catalog >>> -i.e., a runtime generated mosaic-) >>> >>> Everything works well as long as deferred loading of tiles is not >>> used. To force that, I must call RenderedImage.getData() before >>> returning the image resulting from the read operation. That is a >>> tiled one matching the ArcSDE raster tile dimension. >>> >>> But if I don´t do so, strange artifacts appear in the WMS GetMap >>> responses. Some seem to be avoided if I forcedly disable MetaTiling, >>> like in commenting out the following code at GetMapReponse line 165: >>> //if (MetatileMapProducer.isRequestTiled(request, delegate)) {... >>> but others persist. >>> Artifacts are either tiles with repeating patterns (like in a single >>> tile being repeated) or images with blank rectangles. >>> >>> I suspected of the ESRI api's thread safety wrt internal tile buffers >>> or something, but that might happen even for non tiling requests. >>> >>> So, (and apologies for the long preface), question is, does someone >>> already experienced this kind of problems with JAI and deferred >>> loading? Any known work around or thing to have in mind to get it >>> working properly? >>> >>> I know it's hard to answer with such a vague description but if you >>> ever experienced something similar please let me know. >>> >>> Cheers, >>> Gabriel >>> >> >> >> ------------------------------------------------------------------------------ >> >> Come build with us! The BlackBerry(R) Developer Conference in SF, CA >> is the only developer event you need to attend this year. Jumpstart your >> developing skills, take BlackBerry mobile applications to market and >> stay ahead of the curve. Join us from November 9 - 12, 2009. Register >> now! >> http://p.sf.net/sfu/devconference >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Geoserver-devel mailing list >> Geoserver-devel@... >> https://lists.sourceforge.net/lists/listinfo/geoserver-devel >> > > -- Gabriel Roldan OpenGeo - http://opengeo.org Expert service straight from the developers.-
From: Daniele Romagnoli
Subject: Re: [Geoserver-devel] looks like a jpeg bug? Re: weird image artifacts with JAI deferred loading
Date: Oct 29, 2009 06:47 AM
Hi Gabriel, sorry but I don't remember a similar issue before. (But my memory could fail...) Simone could have more info about this but actually he is away for vacation. Regards, Daniele On Wed, Oct 28, 2009 at 9:18 PM, Gabriel Roldan <groldan@...> wrote: > Hey Arne, that´s good to know... so it´s not me getting crazy, but a known > issue. > > May be Simone or Daniele know something about that? > > Cheers, > Gabriel > > > Arne Kepp wrote: > >> I've had this same problem in GWC[1], it's been around for 2 years ++, >> never understood how it could work in GeoServer, but then again I'm not sure >> I ever configured a tiled=true client correctly either. (In my code useJAI >> is set to false if it's JPEG) >> >> -Arne >> >> 1: >> http://geowebcache.org/trac/browser/trunk/geowebcache/src/main/java/org/geowebcache/layer/wms/WMSMetaTile.java#L223 >> >> >> Gabriel Roldan wrote: >> >>> Hey, >>> >>> wrt the repeated tiles when metatiling is enabled, I found it only >>> happens if the output format is JPEG, PNG and GIS are ok. >>> >>> The attached images show how tiling requests look when metatiling is >>> activated or not, and output format is jpeg. >>> >>> any idea? >>> >>> Cheers, >>> Gabriel >>> Gabriel Roldan wrote: >>> >>>> Hey, >>>> >>>> as you might know, the ArcSDE raster support is now based on JAI, like >>>> in using a custom ImageInputStream to read from a SDE raster query stream. >>>> For background: this ImageInputStream backs a RawImageInputStream. >>>> >>>> So the resulting RenderedImage for an ArcSDE coverage request is a small >>>> JAI ops chain like read->crop->scale->translate->mosaic (forget about the >>>> mosaic part when the arcsde raster is not a raster catalog -i.e., a runtime >>>> generated mosaic-) >>>> >>>> Everything works well as long as deferred loading of tiles is not used. >>>> To force that, I must call RenderedImage.getData() before returning the >>>> image resulting from the read operation. That is a tiled one matching the >>>> ArcSDE raster tile dimension. >>>> >>>> But if I don´t do so, strange artifacts appear in the WMS GetMap >>>> responses. Some seem to be avoided if I forcedly disable MetaTiling, like in >>>> commenting out the following code at GetMapReponse line 165: >>>> //if (MetatileMapProducer.isRequestTiled(request, delegate)) {... >>>> but others persist. >>>> Artifacts are either tiles with repeating patterns (like in a single >>>> tile being repeated) or images with blank rectangles. >>>> >>>> I suspected of the ESRI api's thread safety wrt internal tile buffers or >>>> something, but that might happen even for non tiling requests. >>>> >>>> So, (and apologies for the long preface), question is, does someone >>>> already experienced this kind of problems with JAI and deferred loading? Any >>>> known work around or thing to have in mind to get it working properly? >>>> >>>> I know it's hard to answer with such a vague description but if you ever >>>> experienced something similar please let me know. >>>> >>>> Cheers, >>>> Gabriel >>>> >>>> >>> >>> ------------------------------------------------------------------------------ >>> >>> Come build with us! The BlackBerry(R) Developer Conference in SF, CA >>> is the only developer event you need to attend this year. Jumpstart your >>> developing skills, take BlackBerry mobile applications to market and stay >>> ahead of the curve. Join us from November 9 - 12, 2009. Register now! >>> http://p.sf.net/sfu/devconference >>> ------------------------------------------------------------------------ >>> >>> _______________________________________________ >>> Geoserver-devel mailing list >>> Geoserver-devel@... >>> https://lists.sourceforge.net/lists/listinfo/geoserver-devel >>> >>> >> >> >> > > -- > Gabriel Roldan > OpenGeo - http://opengeo.org > Expert service straight from the developers. > -- ------------------------------------------------------- Eng. Daniele Romagnoli Software Engineer GeoSolutions S.A.S. Via Carignoni 51 55041 Camaiore (LU) Italy phone: +39 0584983027 fax: +39 0584983027 mob: +39 328 0559267 http://www.geo-solutions.it -------------------------------------------------------
-
-
From: groldan
Subject: Re: [Geoserver-devel] looks like a jpeg bug? Re: weird image artifacts with JAI deferred loading
Date: Oct 29, 2009 02:25 PM
Hi Arne, Looks like the same workaround needs to be done in GeoServer, this time in JPEGMapProducer. Patch for GeoServer at <http://jira.codehaus.org/browse/GEOS-3612>. Arne Kepp wrote: > I've had this same problem in GWC[1], it's been around for 2 years ++, > never understood how it could work in GeoServer, but then again I'm not > sure I ever configured a tiled=true client correctly either. (In my code > useJAI is set to false if it's JPEG) > > -Arne > > 1: > http://geowebcache.org/trac/browser/trunk/geowebcache/src/main/java/org/geowebcache/layer/wms/WMSMetaTile.java#L223 > > > > Gabriel Roldan wrote: >> Hey, >> >> wrt the repeated tiles when metatiling is enabled, I found it only >> happens if the output format is JPEG, PNG and GIS are ok. >> >> The attached images show how tiling requests look when metatiling is >> activated or not, and output format is jpeg. >> >> any idea? >> >> Cheers, >> Gabriel >> Gabriel Roldan wrote: >>> Hey, >>> >>> as you might know, the ArcSDE raster support is now based on JAI, >>> like in using a custom ImageInputStream to read from a SDE raster >>> query stream. For background: this ImageInputStream backs a >>> RawImageInputStream. >>> >>> So the resulting RenderedImage for an ArcSDE coverage request is a >>> small JAI ops chain like read->crop->scale->translate->mosaic (forget >>> about the mosaic part when the arcsde raster is not a raster catalog >>> -i.e., a runtime generated mosaic-) >>> >>> Everything works well as long as deferred loading of tiles is not >>> used. To force that, I must call RenderedImage.getData() before >>> returning the image resulting from the read operation. That is a >>> tiled one matching the ArcSDE raster tile dimension. >>> >>> But if I don´t do so, strange artifacts appear in the WMS GetMap >>> responses. Some seem to be avoided if I forcedly disable MetaTiling, >>> like in commenting out the following code at GetMapReponse line 165: >>> //if (MetatileMapProducer.isRequestTiled(request, delegate)) {... >>> but others persist. >>> Artifacts are either tiles with repeating patterns (like in a single >>> tile being repeated) or images with blank rectangles. >>> >>> I suspected of the ESRI api's thread safety wrt internal tile buffers >>> or something, but that might happen even for non tiling requests. >>> >>> So, (and apologies for the long preface), question is, does someone >>> already experienced this kind of problems with JAI and deferred >>> loading? Any known work around or thing to have in mind to get it >>> working properly? >>> >>> I know it's hard to answer with such a vague description but if you >>> ever experienced something similar please let me know. >>> >>> Cheers, >>> Gabriel >>> >> >> >> ------------------------------------------------------------------------------ >> >> Come build with us! The BlackBerry(R) Developer Conference in SF, CA >> is the only developer event you need to attend this year. Jumpstart your >> developing skills, take BlackBerry mobile applications to market and >> stay ahead of the curve. Join us from November 9 - 12, 2009. Register >> now! >> http://p.sf.net/sfu/devconference >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Geoserver-devel mailing list >> Geoserver-devel@... >> https://lists.sourceforge.net/lists/listinfo/geoserver-devel >> > > -- Gabriel Roldan OpenGeo - http://opengeo.org Expert service straight from the developers.-
From: aaime
Subject: Re: Re: [Geoserver-devel] looks like a jpeg bug? Re: weird image artifacts with JAI deferred loading
Date: Oct 29, 2009 08:25 PM
Gabriel Roldan ha scritto: > Hi Arne, > > Looks like the same workaround needs to be done in GeoServer, this time > in JPEGMapProducer. Patch for GeoServer at > <http://jira.codehaus.org/browse/GEOS-3612>. Hi, sorry for the late answer, I'm supposed to be in vacation and I was not actually reading the mails that ended up in OpenGeo related folders (since massgis-devel was cc'ed the mail ended up in one of such folders). Someone reported me a very similar issue during FOSS4G and I did open a bug report, http://jira.codehaus.org/browse/GEOS-3592, managed to talk a little with Simone about it but we did not have time to look into it. The issue is occurring only if one has the native JPEG acceleration enabled but the native JAI libraries are not there. The patch you're proposing is probably going to slow down image serving in the case the JAI Image I/O extensions are actually installed (I need to double check but I believe the bug is not showing itself if you either disable native jpeg acceleration in the GS config or if you properly install the native JPEG acceleration libraries) Cheers Andrea
-
From: groldan
Subject: Re: [Geoserver-devel] Re: looks like a jpeg bug? Re: weird image artifacts with JAI deferred loading
Date: Oct 30, 2009 10:34 AM
Andrea Aime wrote: > Gabriel Roldan ha scritto: >> Hi Arne, >> >> Looks like the same workaround needs to be done in GeoServer, this time >> in JPEGMapProducer. Patch for GeoServer at >> <http://jira.codehaus.org/browse/GEOS-3612>. > > Hi, > sorry for the late answer, I'm supposed to be in vacation and I was > not actually reading the mails that ended up in OpenGeo related > folders (since massgis-devel was cc'ed the mail ended up in one of > such folders). no worries, thanks for replying. I am supposed to be on vacation too, just kept on following this issue as it prevented me to finish out the project last week. > > Someone reported me a very similar issue during FOSS4G and I did open > a bug report, http://jira.codehaus.org/browse/GEOS-3592, managed to > talk a little with Simone about it but we did not have time to look into it. > > The issue is occurring only if one has the native JPEG acceleration > enabled but the native JAI libraries are not there. The patch you're > proposing is probably going to slow down image serving in the case > the JAI Image I/O extensions are actually installed (I need to double > check but I believe the bug is not showing itself if you either > disable native jpeg acceleration in the GS config or if you properly > install the native JPEG acceleration libraries) That makes sense I think, as I'm running on Mac I don't have native image-io at all, only JAI. I am aware the patch might slow down things hence the request for support. I am pretty sure the bug shows itself anyways, at least in my case. let's follow up on it when we're all back from vacation. Cheers, Gabriel > > Cheers > Andrea > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry(R) Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9 - 12, 2009. Register now! > http://p.sf.net/sfu/devconference > _______________________________________________ > Geoserver-devel mailing list > Geoserver-devel@... > https://lists.sourceforge.net/lists/listinfo/geoserver-devel -- Gabriel Roldan OpenGeo - http://opengeo.org Expert service straight from the developers.
-
From: Simone Giannecchini
Subject: Re: [Geoserver-devel] Re: looks like a jpeg bug? Re: weird image artifacts with JAI deferred loading
Date: Nov 10, 2009 12:51 PM
Just to confirm, do you need any help for GEOS-3612 and GEOS-3592, my understanding is that you do not? Simone. ------------------------------------------------------- Ing. Simone Giannecchini GeoSolutions S.A.S. Founder - Software Engineer Via Carignoni 51 55041 Camaiore (LU) Italy phone: +39 0584983027 fax: +39 0584983027 mob: +39 333 8128928 http://www.geo-solutions.it http://geo-solutions.blogspot.com/ http://simboss.blogspot.com/ http://www.linkedin.com/in/simonegiannecchini ------------------------------------------------------- On Fri, Oct 30, 2009 at 1:25 AM, Andrea Aime <aaime@...> wrote: > Gabriel Roldan ha scritto: >> Hi Arne, >> >> Looks like the same workaround needs to be done in GeoServer, this time >> in JPEGMapProducer. Patch for GeoServer at >> <http://jira.codehaus.org/browse/GEOS-3612>. > > Hi, > sorry for the late answer, I'm supposed to be in vacation and I was > not actually reading the mails that ended up in OpenGeo related > folders (since massgis-devel was cc'ed the mail ended up in one of > such folders). > > Someone reported me a very similar issue during FOSS4G and I did open > a bug report, http://jira.codehaus.org/browse/GEOS-3592, managed to > talk a little with Simone about it but we did not have time to look into it. > > The issue is occurring only if one has the native JPEG acceleration > enabled but the native JAI libraries are not there. The patch you're > proposing is probably going to slow down image serving in the case > the JAI Image I/O extensions are actually installed (I need to double > check but I believe the bug is not showing itself if you either > disable native jpeg acceleration in the GS config or if you properly > install the native JPEG acceleration libraries) > > Cheers > Andrea > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry(R) Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9 - 12, 2009. Register now! > http://p.sf.net/sfu/devconference > _______________________________________________ > Geoserver-devel mailing list > Geoserver-devel@... > https://lists.sourceforge.net/lists/listinfo/geoserver-devel >
-
From: groldan
Subject: Re: [Geoserver-devel] Re: looks like a jpeg bug? Re: weird image artifacts with JAI deferred loading
Date: Nov 10, 2009 02:00 PM
Hi Simone, thanks for asking. Please read bellow. Simone Giannecchini wrote: > Just to confirm, > do you need any help for GEOS-3612 and GEOS-3592, my understanding is > that you do not? I actually do. My understanding of the problem is as follows: - this happens only on trunk (2.0.x too I assume), but not on 1.7.x - problem is the JPEG writer, or rather the ImageWorker.writeJPEG, does not respect the image location and always writes the image starting at 0,0 - this shows up when doing a tiling request, since the metatile creates each tile as a subset of the metatile image. Hence the tiles being RenderedImages with x,y origin not at 0,0 suffer from this - The following block of code in JPEGMapProducer does not seem to be causing any effect: if (! jpegNativeAcc&&image.getMinX()!=0 || image.getMinY()!=0) { // I don't make an expliciti copy anymore, hoping that sooner or later: // A> this bug will be fixed // B> JDK JpegImageWriter will stop making a straight copy of the input raster mage= TranslateDescriptor.create(image, new Float(-image.getMinX()), new Float(-image.getMinY()), new InterpolationNearest(), null); } the JPEG writer keeps on writing the tile starting at 0,0 of the backing image (the metatile one). So any advice on how to better work around this, if possible, than the patch I've attached to GEOS-3612 would be very much welcomed, as that is expected to be imposing some performance penalty, though I didn't measure it. Cheers, Gabriel-
From: Simone Giannecchini
Subject: Re: [Geoserver-devel] Re: looks like a jpeg bug? Re: weird image artifacts with JAI deferred loading
Date: Nov 27, 2009 02:15 AM
Problem should have been fixed on trunk and 2.0.x. Please, test. Simone. ------------------------------------------------------- Ing. Simone Giannecchini GeoSolutions S.A.S. Founder - Software Engineer Via Carignoni 51 55041 Camaiore (LU) Italy phone: +39 0584983027 fax: +39 0584983027 mob: +39 333 8128928 http://www.geo-solutions.it http://geo-solutions.blogspot.com/ http://simboss.blogspot.com/ http://www.linkedin.com/in/simonegiannecchini ------------------------------------------------------- On Tue, Nov 10, 2009 at 7:00 PM, Gabriel Roldan <groldan@...> wrote: > Hi Simone, > > thanks for asking. Please read bellow. > > Simone Giannecchini wrote: >> >> Just to confirm, >> do you need any help for GEOS-3612 and GEOS-3592, my understanding is >> that you do not? > > I actually do. > My understanding of the problem is as follows: > - this happens only on trunk (2.0.x too I assume), but not on 1.7.x > - problem is the JPEG writer, or rather the ImageWorker.writeJPEG, does not > respect the image location and always writes the image starting at 0,0 > - this shows up when doing a tiling request, since the metatile creates each > tile as a subset of the metatile image. Hence the tiles being RenderedImages > with x,y origin not at 0,0 suffer from this > - The following block of code in JPEGMapProducer does not seem to be causing > any effect: > if (! jpegNativeAcc&&image.getMinX()!=0 || image.getMinY()!=0) { > // I don't make an expliciti copy anymore, hoping that sooner or later: > // A> this bug will be fixed > // B> JDK JpegImageWriter will stop making a straight copy of the input > raster > mage= TranslateDescriptor.create(image, new Float(-image.getMinX()), new > Float(-image.getMinY()), new InterpolationNearest(), null); > } > > the JPEG writer keeps on writing the tile starting at 0,0 of the backing > image (the metatile one). > > So any advice on how to better work around this, if possible, than the patch > I've attached to GEOS-3612 would be very much welcomed, as that is expected > to be imposing some performance penalty, though I didn't measure it. > > Cheers, > Gabriel > > > -- > Archive: http://lists.opengeo.org/massgis-dev/archive/2009/11/1257879627043 > To unsubscribe send an email with subject "unsubscribe" to > massgis-devel@.... Please contact > massgis-devel-manager@... for questions. > >
-
-
-
-
-
To unsubscribe from this list send an email to the list with the subject "unsubscribe" and an empty body.
metatiling_disabled.jpg (image/jpeg) 254.8 kB
text.html (text/html) 5.4 kB