Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Further Node-related config related to database names #278

Closed
2 of 3 tasks
brettz9 opened this issue Jan 20, 2017 · 1 comment
Closed
2 of 3 tasks

Further Node-related config related to database names #278

brettz9 opened this issue Jan 20, 2017 · 1 comment
Assignees
Labels
Milestone

Comments

@brettz9
Copy link
Collaborator

brettz9 commented Jan 20, 2017

The following are further todos related to #274 which which are lower priority since if someone really needs the Unicode, they can override escapeDatabaseName themselves (though it'd admittedly be better to bake in an option) and the third issue is not really bug-related:

  • Striking out as may not be needed: Provide an option for (Node) users to escape (and potentially unescape) upper-case letters when mapping case-sensitive IndexedDB database names to SQLite database file names since a number of modern Unicode file systems that might host such SQLite database files are nevertheless case-insensitive and thus a system allowing arbitrary (finite-length) strings could inadvertently provide a conflict if relying purely on string equality. Currently we are only escaping A-Z.
    • Case-insensitivity may only actually be a problem for ASCII at least in Windows. I wonder if the other modern systems referenced above have similar issues.
    • If we do find a need for it:
      • If we escape other characters, we'd need to use another numbered escape sequence like ^2 or ^3.
      • If using a code point escape sequence, we'd need to indicate a different length following like 4 or 6 (the other characters will be more than 2 hex digits)
      • We could alternatively map to a form of lower-case
        • toLocaleLowerCase would risk portability problems, and though toLowerCase is Unicode-aware (apparently using the C and S case foldings), we want not only locale-independent and moderately human-friendly forms for escaping, but also round-trippable ones so that if a database file is transported to another system, it can be unescaped and understood.
        • The repository for the latest Unicode (currently unicode-9.0.0/Case_Folding) would help a lot with retrieval of the case mappings (Our file-system-safe escape character, ^ could be used along with the lower-case letter as well a number (where needed) to avoid a potential round-tripping conflict (e.g., with ^k1 and ^k2 to escape upper case K and the Kelvin sign, respectively, as they would otherwise both map back to the letter "K", or perhaps better for readability ^k and ^3k2 with 3 indicate a non-ASCII (or lower code point) case mapping and 2 indicating it was the 2nd possible conflict; this could become fragile with new Unicode versions, however.)
        • If adding this support (via XRegExp if Babel/ES doesn't support case matching by then), we may as well also change our hard-coding of Unicode ID Start, etc. to use XRegExp's also.
  • Perhaps we should actually be more concerned about escaping characters which might be normalized by NFD in the MacOS if this is accurate: https://en.wikipedia.org/wiki/Filename#Unicode_interoperability as that could also provide uniqueness issues
    • This could be made easier with building mappings based on String.prototype.normalize and/or the polyfill
    • Also see unicode-9.0.0/Binary_Property/Expands_On_ (NFC/NFD/NFKC/NFKD); might just escape NFD-expanding ones for compatibility with MacOS
  • Offer config for Node users to leverage node-sqlite3's (and consequently node-websql's) ability to use empty string or ":memory:" in-memory databases, including "file::memory:..." URLs for creating temporary databases (should also get this latter type documented on node-websql). We might consider keeping track of memory databases to allow closing them via deleteDatabase (though would need to return some kind of unique ID)? Add close(function (err) {}) callback to node-sqlite3 (websqlOpenDatabaseDb._db.close()) calls; https://github.com/mapbox/node-sqlite3/wiki/API#databaseclosecallback ; closing will also auto-delete empty string temp. dbs per https://sqlite.org/inmemorydb.html (also to call when deleting or closing/finishing connection and reporting errors as per Utilize SQLite close #295 )
  • Option in Node to delete the actual database file as well and delete __sys__.sqlite database if empty (may need queue (or wholly sync behavior--see Provide optional Node support for accurate task/microtask behavior #296 ) to avoid errors after deleteDatabase followed by open) whether after deleting database or aborting an upgrade
@brettz9 brettz9 self-assigned this Jan 20, 2017
@brettz9 brettz9 added this to the 4.0.0 milestone Jan 20, 2017
@brettz9 brettz9 added the Bug label Jan 26, 2017
brettz9 added a commit to brettz9/IndexedDBShim that referenced this issue Apr 8, 2017
    `normalizeDatabaseNames` to escape characters that expand on NFD
    normalization so as to avoid accidental loss of Unicode expression and
    potential name clashes on MacOS which does NFD normalization on the file
    system (user should normalize themselves if they want normalization);
    part of indexeddbshim#278
brettz9 added a commit to brettz9/IndexedDBShim that referenced this issue Apr 8, 2017
…baseNames` to

    escape characters that expand on NFD normalization so as to avoid
    accidental loss of Unicode expression and potential name clashes on MacOS
    which does NFD normalization on the file system (user should normalize
    themselves if they want normalization); part of indexeddbshim#278
brettz9 added a commit to brettz9/IndexedDBShim that referenced this issue Apr 8, 2017
    escape characters that expand on NFD normalization so as to avoid
    accidental loss of precise Unicode code points and thus potential name
    clashes on the Mac which does NFD normalization on the file system (user
    should normalize themselves if they want normalization); part of indexeddbshim#278
- Testing (Mocha, fake, mock, W3C Old, QUnit): Avoid adding ".sqlite" prefix manually
    as now adding automatically by default
brettz9 added a commit to brettz9/IndexedDBShim that referenced this issue Apr 9, 2017
- Enhancement: Make addition of `.sqlite` extension to `__sysdb__` conditional
    on `CFG.addSQLiteExtension` setting
- Enhancement: Support `autoName` config to interpret empty string name as a
    cue for creating a database name automatically (introspect on
    `IDBDatabase.name` to get the actual name used) (untested)
- Enhancement: Support `memoryDatabase` config to cause all opening, deleting,
    and listing to be of SQLite in-memory databases (for Node); name supplied
    by user is still used (including to automatically build a cache since
    SQLite does not allow naming of in-memory databases); the name is also
    accessible to `IDBFactory.webkitGetDatabaseNames()`; causes database
    name/version tracking to also be within an in-memory database; part of
    indexeddbshim#278 (untested); builds on work by @ThomasGreiner)
- Enhancement: Support `cacheDatabaseInstances` config to ensure that any
    repeat `IDBFactory.open` call to the same name and version (assuming
    no deletes or aborts causing rollbacks) will reuse the same SQLite
    `openDatabase` instance
brettz9 added a commit to brettz9/IndexedDBShim that referenced this issue Apr 9, 2017
- Enhancement: Make addition of `.sqlite` extension to `__sysdb__` conditional
    on `CFG.addSQLiteExtension` setting
- Enhancement: Support `autoName` config to interpret empty string name as a
    cue for creating a database name automatically (introspect on
    `IDBDatabase.name` to get the actual name used) (untested)
- Enhancement: Support `memoryDatabase` config to cause all opening, deleting,
    and listing to be of SQLite in-memory databases (for Node); name supplied
    by user is still used (including to automatically build a cache since
    SQLite does not allow naming of in-memory databases); the name is also
    accessible to `IDBFactory.webkitGetDatabaseNames()`; causes database
    name/version tracking to also be within an in-memory database; part of
    indexeddbshim#278 (untested); builds on work by @ThomasGreiner)
- Enhancement: Support `cacheDatabaseInstances` config to ensure that any
    repeat `IDBFactory.open` call to the same name and version (assuming
    no deletes or aborts causing rollbacks) will reuse the same SQLite
    `openDatabase` instance
brettz9 added a commit to brettz9/IndexedDBShim that referenced this issue Apr 9, 2017
- Enhancement: Make addition of `.sqlite` extension to `__sysdb__` conditional
    on `CFG.addSQLiteExtension` setting
- Enhancement: Support `autoName` config to interpret empty string name as a
    cue for creating a database name automatically (introspect on
    `IDBDatabase.name` to get the actual name used) (untested)
- Enhancement: Support `memoryDatabase` config to cause all opening, deleting,
    and listing to be of SQLite in-memory databases (for Node); name supplied
    by user is still used (including to automatically build a cache since
    SQLite does not allow naming of in-memory databases); the name is also
    accessible to `IDBFactory.webkitGetDatabaseNames()`; causes database
    name/version tracking to also be within an in-memory database; part of
    indexeddbshim#278 (untested); builds on work by @ThomasGreiner)
- Enhancement: Support `cacheDatabaseInstances` config to ensure that any
    repeat `IDBFactory.open` call to the same name and version (assuming
    no deletes or aborts causing rollbacks) will reuse the same SQLite
    `openDatabase` instance
brettz9 added a commit to brettz9/IndexedDBShim that referenced this issue Apr 9, 2017
…nditional

    on `CFG.addSQLiteExtension` setting
- Enhancement: Support `cacheDatabaseInstances` config to ensure that any
    repeat `IDBFactory.open` call to the same name and version (assuming
    no deletes or aborts causing rollbacks) will reuse the same SQLite
    `openDatabase` instance
- Enhancement: Support `autoName` config to interpret empty string name as a
    cue for creating a database name automatically (introspect on
    `IDBDatabase.name` to get the actual name used) (untested)
- Enhancement: Support `memoryDatabase` config to cause all opening, deleting,
    and listing to be of SQLite in-memory databases (for Node); name supplied
    by user is still used (including to automatically build a cache since
    SQLite does not allow naming of in-memory databases); the name is also
    accessible to `IDBFactory.webkitGetDatabaseNames()`; causes database
    name/version tracking to also be within an in-memory database; part of
    indexeddbshim#278 (untested); builds on work by @ThomasGreiner)
@brettz9
Copy link
Collaborator Author

brettz9 commented Mar 24, 2020

Tracking the remaining issue for the new package at #11

@brettz9 brettz9 closed this as completed Mar 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant