TODO

[[ Galapix ToDo ]]
_______________________________________________________________________________

Release procedure:
==================

* test that Galapix works with and without SpaceNavigator libraries

* test that all Galapix functions are accessible without SpaceNavigator

* test that mouse wheel is supported

* test svn trunk a lot

* test database compatibilty

* increment the version number

* check that all keyboard shortcuts are documented and sort them a bit to be logical

* svn cp https://galapix.googlecode.com/svn/trunk/galapix@${NUM} tags/galapix-0.1.2

* svn export http://galapix.googlecode.com/svn/tags/galapix-0.1.2


Galapix 0.2.0 Release ToDo:
---------------------------

* check with sqlite3_table_column_metadata() that the table has the
  right format

* need to display the thread status graphically somehow (graphical
  rotating wheel? per image or overall?)

* seperate galapix from the thumbgen code, so that there is a
  galapix-tool and a galapix program

* galapix FILE...
   -h, --help
   -d, --database FILE    Use database FILE
   -p, --pattern PATTERN  Show all files in the database matching PATTERN

* galapix-thumbgen
   -m, --minsize X      Generate tiles till this size
   -d, --database FILE  

* galapix-db [info|list|merge|cleanup|delete]
   -d, --database FILE

  galapix-db merge -d outdatabase.sqlite indatabase1.sqlite indatabase2.sqlite ...
  galapix-db cleanup
  galapix-db delete -d database1.sqlite [PATTERN]...

Bugs and Issues
===============

* "libpng error: Not a PNG file" should not leak out of libpng, should
  be handled by proper logging functions


Galapix 0.1.3 Release ToDo:
===========================

User Interface:
---------------

* refresh fails at high zoom levels (displays wrong tiles)

* rewrite overlap solving to be fast or disable it, as it currently
  just halts the whole app

* fix random layout so that all image sizes are handled equally,
  currently low-res images get really small, also take aspect
  ration into account

* add function to relayout just the selection instead of all images

* make middle button exit trackball mode

* ZoomTool doesn't know about trackball_mode and thus zooms not into
  the center of the picture, but to the mouse cursor


Other Things:
-------------

* always flush database completly in one go (whey do we get multi
  flushes in a row?)

* io kills interactivity (why?)

* (!!!) jpeg's from archive should only generate tiles fro their zoom level,
  generating all kills performance and eats tons of memory (why?)

* we might need a ThreadMessageQueue with a size limit, that blocks when it is full

* create a directory with test images for all supported image and
  archive formats

* double-load of .kra files is still present

  - FileEntry is generated -> send to image
  - Tiles start generating
  - image request tiles
  - new TileGeneration job is started as FileEntryGenerationJob isn't tracked

  Solution: either add tracking of FileEntryGenerationJob or merge it
  back into TileGenerationJob

* add more custom sorter (sort by date, ...)

* need to handle mtime properly (checking is expensive for thousands
  of images), currently mtime is recorded, but not checked on load, at
  least for some files (archives might differ)

* sorting needs to be changed, as a TileProvider doesn't necesarrily
  have a URL (pass URL and TileProvider to Image? Have a
  TileProvider::get_url()?)

* broken images must be detected and marked, so they aren't
  re-requested over and over again

* make thumbnail generation output optional --log all --verbose == --log info

* need to find proper location to filter out Size(0,0) FileEntries


General Problems:
=================

* currently only out-of-screen cancels jobs, but zooming need to cancel jobs too

* need a framework do handle downloads in a multithreaded fashion

* add other gigapixel format support

http://gasi.ch/blog/openzoom-description-format/

http://gigapan.org/gigapans/15374/
var fl_vars = {
 url:"http://tile15.gigapan.org/gigapans0/15374/tiles/",
 suffix:".jpg",
 startHideControls:"0",
 width:"59783",
 height:"24658",
 nlevels:"9",
 cleft:"0",
 ctop:"0",
 cright:"59783.0",
 cbottom:"24658.0",
 startEnabled:"1",
 notifyWhenLoaded:"1",
 startHideWatermark:"1"
};

1 http://tile15.gigapan.org/gigapans0/15374/tiles/r0.jpg
2 http://tile15.gigapan.org/gigapans0/15374/tiles/r00.jpg
3 http://tile15.gigapan.org/gigapans0/15374/tiles/r00/r000.jpg
4 http://tile15.gigapan.org/gigapans0/15374/tiles/r00/r0000.jpg
5 http://tile15.gigapan.org/gigapans0/15374/tiles/r00/000/r00000.jpg
6 http://tile15.gigapan.org/gigapans0/15374/tiles/r00/000/r000000.jpg
7 http://tile15.gigapan.org/gigapans0/15374/tiles/r00/000/r0000000.jpg
8 http://tile15.gigapan.org/gigapans0/15374/tiles/r00/000/r00000000.jpg

- http://tile15.gigapan.org/gigapans0/15374/tiles/r00/000/000/r000000000.jpg

* a separate table for small tiles could speed things up (i.e. either
  just the lowres'est tile or all tiles that contain the whole image)

select count(*) from tiles;
5695025
select count(*) from files;
500126

* including all fields of the tiles table in the tiles_index could
  speed things up 2x, but would require 2x the storage

* sometimes images end up tiny, some race condition in getting image size likely

* deletion of tiles/fileentries is *extremely* slow, maybe grouping could help

* do something to ensure that callbacks always go back to a valid
  object, not an already deleted one

* temp mousegrab while middle button is held for better scrolling

* review the API of SoftwareSurface and clean it up,
  shared_ptr_from_this() could be avoided when static functions are
  used instead of member functions, i.e.

  SoftwareSurfaceFactory::transform(surface, kRot90);
  SoftwareSurfaceFactory::clone(surface);

* very small images seem broken

 TileGenerationJob: have [0..1] generating [-1..-1]: FileId(51152):

* separate animation from Image, maybe even position

galapix.sdl: src/jobs/tile_generation_job.cpp:54: bool TileGenerationJob::request_tile(const JobHandle&, int, const Vector2i&, const boost::function<void(TileEntry)>&): Assertion `scale < m_min_scale_in_db || scale > m_max_scale_in_db' failed.

* failed loading does not propagate to the image, need some kind of
  callback to report errors

* requests can stack up so that galapix is busy flushing the cache
  multiple times in a row before stuff can continue

* TileEntry::get_blob vs TileEntry::get_surface who uses what?

* figure out where all the memory goes, galapix should ideally need no
  more then what is visible on the screen, which is much less then the
  hundreds of megabyte that ps shows

* memory behaviour:

  after start up: 50MB
  when first picture appear on screen instant jump to 128MB
  low zoom levels don't seem to wast space, only high ones
  zooming in and out also wastes space

* make tile and file cache size configurable

* DatabaseThread is very similar to JobWorkerThread, maybe merge them

* maybe have something like JobHandle->spawn_child_job() -> No

* quality of small thumbnails is very bad, should start with larger
  thumbnails and then halve() them down instead of using scale()

* shared_ptr<> impl is evil

* implement some proper stuff to report the current status (what jobs
  are running, what an image is doing, etc)

* add a is_queued() is_running() to the JobHandle maybe that could be
  useful for something

* use highres tiles when they are in cache when zooming out (currently
  only using one level of highres data)

Graphical User Interface
========================

* implement auto-pilot that navigates images randomly or in sequence
  (i.e. slideshow with a bit of zooming)

* use framebuffer_object to speed up the drawing

* use graphic indicator for loading (alpha flashing)

* add optional fullscreen-sized cross to indicate cursor position

* add two button mode: hold-right button for drag&drop zoom

* when view is rotated, zooming in position is not rotated with it

* add X11 drag&drop support to SDL

* something useless: add puzzle mode that randomly places tiles on the
  screen and let the user put them back together

* write an auto-fit option: take all pictures fully visible on the
  screen, select them then fit them to the screen

* key stuck issues happens: SDL event queue getting filled up? and key-up events lost?

* add raise/lower buttons 

* make 'd' clone the current selection

* add shift-click support to the MoveTool, also rename MoveTool to SelectTool

* add button to flip the view

File Plugins
============

* handle "Premature end of JPEG file" a little better, mark FileEntry
  as broken or so

* Corel CMX Support

* build in a file magic detector: .xcf.bz2 doesn't work for xcfinfo if it ends in .xcf

* check that all file plugins can load from any URL

* move file-extension -> plugin mapping into configuration file

  -> didn't work out, file format handler have to much interdepedency
     to make this practical right now, could wrap the static plugins
     into boost::function or special plugin objects to make it work

* add support for Imagemagick files that create multiple images (i.e. pdf)

* figure out a way to handle multi-image files (multilayer-xcf)
 
  XCF: parse xcfinfo output

  Images are accessed when their width/height is requested and when
  the tiles are generated, everything else is happening outside with
  the database not the files

* check for xcf2pnm, xcfinfo and other tool binaries on startup

* catch PNG read errors properly 

Database
========

* Cleanup higher zoom levels of jpg files (takes hours to run):

  delete from tiles where rowid in (
    select tiles.rowid from files,tiles
       where files.fileid = tiles.fileid AND files.url glob "*.jpg" AND tiles.scale = 0
  );

* Cleanup stuff from the database (quite slow):

  Slow:
  delete from files where url glob "file:///tmp/*";
  delete from tiles where fileid not in (select fileid from files);

  Faster(?):
  delete from tiles where fileid in (select fileid from files where url glob "file:///tmp/*");
  delete from files where url glob "file:///tmp/*";

* add:

   galapix.sdl split -d infile.sqlite -p "glob-patern" outfile.sqlite

 manual way to split the database:

# Create the tables
PRAGMA auto_vacuum = 1;
CREATE TABLE files (fileid INTEGER PRIMARY KEY AUTOINCREMENT,url TEXT UNIQUE, size INTEGER, mtime INTEGER, width INTEGER, height INTEGER);
CREATE UNIQUE INDEX files_index ON files ( url );
CREATE TABLE tiles (fileid INTEGER, scale INTEGER, x INTEGER, y INTEGER, data BLOB, quality INTEGER, format INTEGER);
CREATE INDEX tiles_index ON tiles ( fileid, x, y, scale );

# Attach the orignal
ATTACH "/home/ingo/.galapix/cache.sqlite" as galapix;

# Copy the data
INSERT INTO files select * from galapix.files where url glob "*comics/*";
INSERT INTO tiles select * from galapix.tiles where fileid in (select fileid from files);

* Table convert:
  create table if not exists files_new (fileid INTEGER PRIMARY KEY  AUTOINCREMENT, url TEXT UNIQUE, size  INTEGER,  mtime     INTEGER, width INTEGER, height INTEGER);
  insert into files_new (fileid, url, size, mtime, width, height) select fileid, filename, filesize, mtime, width, height from files;
  alter table files rename to files_old
  alter table files_new rename to files;

* delete tile duplicates via:

DELETE FROM tiles WHERE rowid IN 
(SELECT b.rowid 
 FROM tiles AS a, tiles AS b 
 WHERE a.fileid = b.fileid AND 
            a.x = b.x      AND
            a.y = b.y      AND 
        a.scale = b.scale  AND 
        a.rowid < b.rowid);

Then shrink via (do this in a seperate step, since it requires plenty
of diskspace, 2x(!) times what the database uses):

VACUUM;


* add --size option to 'galapix export'

* There seems to be a conflict between libboost-thread1.34 and
  libboost-thread1.35 (thread joinable(), maybe others)

* tile_cache order is wrong

* use boost::bind() objects instead of custum ones in DatabaseThread

* seperate files and images tables so that information about archive
  files can be saved and other stuff

* when a file is given as URL and refers to an archive it does not
  work from command line, since URLs are assumed to refer to images

* always process get requests to the database before store requests

* add command line option to not follow links and/or archive files

* Write a tempfile class that creates a tempfile from a Blob in cases
  where a command line tool can't read from stdin or from /dev/stdin,
  because it might depend on the file extension to figure out a type
  (theoretical, don't think any app needs that at the moment)

* make SpaceNavigator a seperate thread, do not poll events, but wait
  for them, use SDL_PushEvent to push data as user event

* write grid traversal thing so that we can detect the tiles to draw even on scaled or rotated images

* add layout option that makes use of treemaps:
  http://www.cs.umd.edu/hcil/treemap-history/

* make keyboard bindings configurable

* implement some fun recursive layout where images get constantly smaller

* add tool to cut an image (simple rectangle or more complex polygonal shapes)

* PRAGMA temp_store_directory = '/home/ingo/.galapix/tmp/'; 

* play around with attaching databases to copy stuff around for compability

* is it worth to have a get_size(filename) command? or is it better to
  wait for a proper thumbnail to be there?
  
  -> get_size() has not much benefit, only real benefit is that you
     can layout before a thumbnail is ready, but placeholder rects are
     not pretty and it only works for a few formats anyway, so drop
     and get the smallest possible thumbnail instead and only place
     stuff on the workspace when it is ready

* Add glob pattern URL support: glob:://file:///resources/flickr/* (what would that be good for?!)

* merging databases causes reencoding of the JPEG data due to
  TileEntry storing SoftwareSurface instead of Blob

* implement: Blob Blob::wrap(void* data, int len);

* mouse behaves incorrectly when view is rotated: zooming in doesn't
  zoom in on mouse_pos properly

* Exec::stdout isn't a Blob, cleanup relationship between Blob,
  std::vector<char> and std::vector<uint8_t*> so that we don't need unneeded copies

* need to filter out directories when generating file list for .rar
  (how? since rar doesn't tell you if its a directiory -> use file extension)

* need a way to handle 'maybe' situations, i.e. impossible to tell
  quickly if a URL exist or not, Size is impossible to tell in a quick
  way often as well

* add something to reconstruct a file from the database (as backup in
  case of file loss) (partly done, see 'export')

* make sure that PNM class can handle all PNM formats that XCF can output

* leaving fullscreen doesn't work properly, glViewport size ends up
  being different from the window size

* when relayout track the currently selected item with the camera, so
  that you can switch between random layout and sorted layout without
  losing track of a picture

* some formats (xcf.bz2, stuff in archives) are very slow when it
  comes to getting width/height, would be faster to not get it and
  only get it when generating tiles

* switch to plain Magick libraries, not the C++ version

* add support for checkboard pattern and stuff in the background

* change Framebuffer/Surface so that we can draw images with alpha

* implemented 2x2 Matrix

* implement image rotation

* When thumbgen is running while galapix view is started:

terminate called after throwing an instance of 'SQLiteError'
  what():  SQLiteStatement::~SQLiteStatement: column filename is not unique
Aborted

* support for additional image formats would be nice, via ImageMagicks
  "convert" as a fallback

Galapix Video Demo:
-------------------

http://video.google.com/videoplay?docid=7578014083503114514&hl=en

* create a demo video and publish on Google Video
  - demonstrate the direct zoom into thumbnails
  - demonstrate the view rotation
  - demonstrate relayouting of images
  - demonstrate random layout
  - demonstrate the movement of images
  - demonstrate the resize of images
  - demonstrate isolation and deletion of images

* make sure that it isn't as blurry as the last one and doesn't have
  the framerate glitch

* use a wider range of pictures

* use Big Buck Bunny

// Get a list of all tiles for a given file
// SELECT files.fileid,files.filename,tiles.scale,tiles.x,tiles.y 
//   FROM files, tiles 
//   WHERE files.filename = 'file:///tmp/ssc2008-11a14.jpg' AND files.fileid = tiles.fileid;
// When no tiles are available nothing is returned, which makes this call kind of useless for the intended purpose

* added toggle for GL_LINEAR/GL_NEAREST

* seperate loading of tiles from drawing, making loading/job-cancel
  event based instead of brute force as it is now

* generate all tiles at once for stuff that doesn't support
  incremental loading (every other format then jpeg and jpegs loaded
  from archive files) [mostly done, except for jpegs in archive files]

* check the new features some more

* handle file failures more gracefully, currently we run into a 'job
  rejected' situation because the job can never be completed

* Loading JPEG file can throw exception and that isn't handled
  - failure modes: file not found, data corrupt
_______________________________________________________________________________

* maybe limit the scrolling area, so that one can't scroll completly
  out of the picture: calculate the center of all images and let an
  arrow point into the right direction when no images are visible on
  the screen

* improve overlap constrain solver, make it fast and make it animated

* images are always requested in the highest quality, instead lower
  quality should be requested first and heigher quality only if the
  image is visible for longer periodes of time on the screen

* SDL returns different mouse button numbers depending in if a grab is
  active or not, so buttons will get swapped in left-hand situations

* add more command line options:

  --aspect
  --select (?)

* save color or a 8x8 thumbnail in the FileEntry, for faster loading,
  also save 1x1 Color in FileEntry

* store the max_tile size into the FileTable (should we? We can calc
  it dynamically)

* opening a large directory causes file entries to be generated, but
  no tiles to be shown, because tiles are last in the queue, different
  jobs should have different queue

* Displaying 75'000 images at once gets really slow, OpenGL
  bottleneck, texture packing might help, sending vertex array instead
  of glBegin/glEnd of course too (unlikely that anything of that will
  be enough -> framebuffer should be used to only redraw the needed parts)

* implement smooth scale in SoftwareSurface (like LINEAR instead of NEAREST),

* implement non-copying crop in SoftwareSurface (use parent image data
  as pixel source)

* multi-threading in the tile generator for multicore

* faster incremental tile generation

* File stuff should really be seperate from Tile getting -> Seperate
  worker thread?


Database Stuff:
===============

* There is no handling of tile collisions in the database, so one
  might end up with duplicate entries for the same tile. Tile
  generation is also not optimal, since some data gets discarded.

* Support for multiple databases at once would be good, since a
  database can be used/abused as image file


Thread Stuff:
=============

* Jobs are currently not prioritized properly so unimportant ones
  might be executed before important ones, not harmfull, but its
  unpleasing to see images on the edge of the screen being generated
  before images in the center of the screen.

* request lower res tiles, before requesting highres tiles

* move JPEG decoding in a seperate thread so that the database thread
  can run free with database stuff


Possible Future Improvements:
=============================

* crazy stuff: extend Exec() so that stdout from one Exec() can be
  redirected into another Exec(), could be a bit tricky to get the
  whole archive extraction to connected to the file extraction.

* float isn't enough, on high zoom level it breaks apart, might need double

* implement shader to change hue, saturation, contrast, brightness, etc.

* Toolpalette: Select picture, scale picture, move pictures, align
  selected picture into a group, maybe have a box into which one can
  drop pictures.

* When zooming into an images beyond its original size one will see
  noticable seams at the areas where the tiles meet. This is an
  artifact caused by OpenGLs linear filtering that can't work across
  tile borders, but only on a single tile. A possible fix for this
  would be to add a 1px width border to the tiles, but that would slow
  down the tile generation process from JPEG data, so it is unclear if
  it ever will be implemented.

* galapix completly ignores ~/.thumbnails/, it should maybe make use of
  that to speed up initial thumbnail generation

* Some obscure JPEG formats are not supported:
  Generating tiles for /resources/flickr/flickr/farm3.static.flickr.com/2123/1782745732_d7f5c8cc8c_o.jpg
  Exception: JPEG: Unsupported color depth: 4

* allow rotation of images

* record how long an image has been viewed and allow to display which
  images have been viewed most

* support for greyscale in SoftwareSurface (not really important)


Random Notes:
=============

* jpegtran can't do crop of large scale JPEGs in small amount of
  space, so cropping with libjpeg likely won't work or get more
  complicated

* a single 2048x2048 texture can hold 65536 8x8 tiles -> ~12MB

* when 8x8 or smaller JPEG is larger then RAW, else JPEG always wins

8x8   -> 250
16x16 -> 430
32x32 -> 700
64x64 -> 1600

1920x1080: (64x64) 30x17=510, (32x32) 60x34=2040, (16x16) 120x68=8160, (4x4) 480x270=129600
~100MB for fullscreen 16x16 Thumbs
~32 textures for 129600 thumbs

16x16 -> 1KB
1000 thumbnails -> 1MB
10'000 thumbnails -> 10MB
100'000 thumbnails -> 100MB (graphic card gets full) -> Solution: smaller thumbnails, also unload from GFX card, but keep software backstore

_______________________________________________________________________________


# EOF #