Image support in IDataView and transforms. #489

Closed
Closed
Image support in IDataView and transforms.#489
Assignees
Ivanidzo4ka
Labels
APIIssues pertaining the friendly APIenhancementNew feature or request
Milestone
0718

Description

@Ivanidzo4ka
Ivanidzo4ka
opened on Jul 3, 2018

Currently only way to work with images is by manually loading pixels arrays into array field in the class and wrap that class into collection data source. Which is not pleasant.

I propose to add support for type Image in DataView type system which can be based on System.Drawing.Bitmap This way user can specify Bitmap image in their class and it will be properly consumed by CollectionDataSource.

Also I suggest to implement transform which would accept column with file paths and load this images during pipeline execution. Which allow you to save memory in case of streaming dataview, some trainers support streaming structure of data, and this way you need to keep in memory only one image for current row instead of images for all rows.

Since quite often you need to modify images (Re-scale them, transform image into Grayscale, etc) it make sense to keep Image format as "image" and not directly convert it into vector of floats which we require for learners. Which means we need transform which would convert Image type into feature vector.

To summarize I propose to create following:

Image type based on Bitmap.
ImageLoader transform
ImageResizer transform
ImageGrayscale transform
ImageToPixels transform

@shauheen@TomFinley@glebuk

Metadata

Metadata

Assignees

Labels

APIIssues pertaining the friendly APIenhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
     blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

    Skip to content
    Image support in IDataView and transforms. #489
    Closed
    Closed
    Image support in IDataView and transforms.#489
    Assignees
    Ivanidzo4ka
    Labels
    APIIssues pertaining the friendly APIenhancementNew feature or request
    Milestone
    0718
    Description
    @Ivanidzo4ka
    Ivanidzo4ka
    opened on Jul 3, 2018
    Currently only way to work with images is by manually loading pixels arrays into array field in the class and wrap that class into collection data source. Which is not pleasant.
    I propose to add support for type Image in DataView type system which can be based on System.Drawing.Bitmap This way user can specify Bitmap image in their class and it will be properly consumed by CollectionDataSource.
    Also I suggest to implement transform which would accept column with file paths and load this images during pipeline execution. Which allow you to save memory in case of streaming dataview, some trainers support streaming structure of data, and this way you need to keep in memory only one image for current row instead of images for all rows.
    Since quite often you need to modify images (Re-scale them, transform image into Grayscale, etc) it make sense to keep Image format as "image" and not directly convert it into vector of floats which we require for learners. Which means we need transform which would convert Image type into feature vector.
    To summarize I propose to create following:
    Image type based on Bitmap.
    
ImageLoader transform
    
ImageResizer transform
    
ImageGrayscale transform
    
ImageToPixels transform
    @shauheen@TomFinley@glebuk
    Metadata
    Metadata
    Assignees
    Labels
    APIIssues pertaining the friendly APIenhancementNew feature or request
    Type
    No type
    Projects
    No projects
      Milestone
      Relationships
      None yet
      Development
      No branches or pull requests
      Issue actions
      
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

      Skip to content
      Image support in IDataView and transforms. #489
      Closed
      Closed
      Image support in IDataView and transforms.#489
      Assignees
      Ivanidzo4ka
      Labels
      APIIssues pertaining the friendly APIenhancementNew feature or request
      Milestone
      0718
      Description
      @Ivanidzo4ka
      Ivanidzo4ka
      opened on Jul 3, 2018
      Currently only way to work with images is by manually loading pixels arrays into array field in the class and wrap that class into collection data source. Which is not pleasant.
      I propose to add support for type Image in DataView type system which can be based on System.Drawing.Bitmap This way user can specify Bitmap image in their class and it will be properly consumed by CollectionDataSource.
      Also I suggest to implement transform which would accept column with file paths and load this images during pipeline execution. Which allow you to save memory in case of streaming dataview, some trainers support streaming structure of data, and this way you need to keep in memory only one image for current row instead of images for all rows.
      Since quite often you need to modify images (Re-scale them, transform image into Grayscale, etc) it make sense to keep Image format as "image" and not directly convert it into vector of floats which we require for learners. Which means we need transform which would convert Image type into feature vector.
      To summarize I propose to create following:
      Image type based on Bitmap.
      
ImageLoader transform
      
ImageResizer transform
      
ImageGrayscale transform
      
ImageToPixels transform
      @shauheen@TomFinley@glebuk
      Metadata
      Metadata
      Assignees
      Labels
      APIIssues pertaining the friendly APIenhancementNew feature or request
      Type
      No type
      Projects
      No projects
        Milestone
        Relationships
        None yet
        Development
        No branches or pull requests
        Issue actions
        
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

        Skip to content
        Image support in IDataView and transforms. #489
        Closed
        Closed
        Image support in IDataView and transforms.#489
        Assignees
        Ivanidzo4ka
        Labels
        APIIssues pertaining the friendly APIenhancementNew feature or request
        Milestone
        0718
        Description
        @Ivanidzo4ka
        Ivanidzo4ka
        opened on Jul 3, 2018
        Currently only way to work with images is by manually loading pixels arrays into array field in the class and wrap that class into collection data source. Which is not pleasant.
        I propose to add support for type Image in DataView type system which can be based on System.Drawing.Bitmap This way user can specify Bitmap image in their class and it will be properly consumed by CollectionDataSource.
        Also I suggest to implement transform which would accept column with file paths and load this images during pipeline execution. Which allow you to save memory in case of streaming dataview, some trainers support streaming structure of data, and this way you need to keep in memory only one image for current row instead of images for all rows.
        Since quite often you need to modify images (Re-scale them, transform image into Grayscale, etc) it make sense to keep Image format as "image" and not directly convert it into vector of floats which we require for learners. Which means we need transform which would convert Image type into feature vector.
        To summarize I propose to create following:
        Image type based on Bitmap.
        
ImageLoader transform
        
ImageResizer transform
        
ImageGrayscale transform
        
ImageToPixels transform
        @shauheen@TomFinley@glebuk
        Metadata
        Metadata
        Assignees
        Labels
        APIIssues pertaining the friendly APIenhancementNew feature or request
        Type
        No type
        Projects
        No projects
          Milestone
          Relationships
          None yet
          Development
          No branches or pull requests
          Issue actions
          
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

          Skip to content
          Image support in IDataView and transforms. #489
          Closed
          Closed
          Image support in IDataView and transforms.#489
          Assignees
          Ivanidzo4ka
          Labels
          APIIssues pertaining the friendly APIenhancementNew feature or request
          Milestone
          0718
          Description
          @Ivanidzo4ka
          Ivanidzo4ka
          opened on Jul 3, 2018
          Currently only way to work with images is by manually loading pixels arrays into array field in the class and wrap that class into collection data source. Which is not pleasant.
          I propose to add support for type Image in DataView type system which can be based on System.Drawing.Bitmap This way user can specify Bitmap image in their class and it will be properly consumed by CollectionDataSource.
          Also I suggest to implement transform which would accept column with file paths and load this images during pipeline execution. Which allow you to save memory in case of streaming dataview, some trainers support streaming structure of data, and this way you need to keep in memory only one image for current row instead of images for all rows.
          Since quite often you need to modify images (Re-scale them, transform image into Grayscale, etc) it make sense to keep Image format as "image" and not directly convert it into vector of floats which we require for learners. Which means we need transform which would convert Image type into feature vector.
          To summarize I propose to create following:
          Image type based on Bitmap.
          
ImageLoader transform
          
ImageResizer transform
          
ImageGrayscale transform
          
ImageToPixels transform
          @shauheen@TomFinley@glebuk
          Metadata
          Metadata
          Assignees
          Labels
          APIIssues pertaining the friendly APIenhancementNew feature or request
          Type
          No type
          Projects
          No projects
            Milestone
            Relationships
            None yet
            Development
            No branches or pull requests
            Issue actions
            
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

            Skip to content
            Image support in IDataView and transforms. #489
            Closed
            Closed
            Image support in IDataView and transforms.#489
            Assignees
            Ivanidzo4ka
            Labels
            APIIssues pertaining the friendly APIenhancementNew feature or request
            Milestone
            0718
            Description
            @Ivanidzo4ka
            Ivanidzo4ka
            opened on Jul 3, 2018
            Currently only way to work with images is by manually loading pixels arrays into array field in the class and wrap that class into collection data source. Which is not pleasant.
            I propose to add support for type Image in DataView type system which can be based on System.Drawing.Bitmap This way user can specify Bitmap image in their class and it will be properly consumed by CollectionDataSource.
            Also I suggest to implement transform which would accept column with file paths and load this images during pipeline execution. Which allow you to save memory in case of streaming dataview, some trainers support streaming structure of data, and this way you need to keep in memory only one image for current row instead of images for all rows.
            Since quite often you need to modify images (Re-scale them, transform image into Grayscale, etc) it make sense to keep Image format as "image" and not directly convert it into vector of floats which we require for learners. Which means we need transform which would convert Image type into feature vector.
            To summarize I propose to create following:
            Image type based on Bitmap.
            
ImageLoader transform
            
ImageResizer transform
            
ImageGrayscale transform
            
ImageToPixels transform
            @shauheen@TomFinley@glebuk
            Metadata
            Metadata
            Assignees
            Labels
            APIIssues pertaining the friendly APIenhancementNew feature or request
            Type
            No type
            Projects
            No projects
              Milestone
              Relationships
              None yet
              Development
              No branches or pull requests
              Issue actions
              
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

              Skip to content
              Image support in IDataView and transforms. #489
              Closed
              Closed
              Image support in IDataView and transforms.#489
              Assignees
              Ivanidzo4ka
              Labels
              APIIssues pertaining the friendly APIenhancementNew feature or request
              Milestone
              0718
              Description
              @Ivanidzo4ka
              Ivanidzo4ka
              opened on Jul 3, 2018
              Currently only way to work with images is by manually loading pixels arrays into array field in the class and wrap that class into collection data source. Which is not pleasant.
              I propose to add support for type Image in DataView type system which can be based on System.Drawing.Bitmap This way user can specify Bitmap image in their class and it will be properly consumed by CollectionDataSource.
              Also I suggest to implement transform which would accept column with file paths and load this images during pipeline execution. Which allow you to save memory in case of streaming dataview, some trainers support streaming structure of data, and this way you need to keep in memory only one image for current row instead of images for all rows.
              Since quite often you need to modify images (Re-scale them, transform image into Grayscale, etc) it make sense to keep Image format as "image" and not directly convert it into vector of floats which we require for learners. Which means we need transform which would convert Image type into feature vector.
              To summarize I propose to create following:
              Image type based on Bitmap.
              
ImageLoader transform
              
ImageResizer transform
              
ImageGrayscale transform
              
ImageToPixels transform
              @shauheen@TomFinley@glebuk
              Metadata
              Metadata
              Assignees
              Labels
              APIIssues pertaining the friendly APIenhancementNew feature or request
              Type
              No type
              Projects
              No projects
                Milestone
                Relationships
                None yet
                Development
                No branches or pull requests
                Issue actions
                
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

                Skip to content
                Image support in IDataView and transforms. #489
                Closed
                Closed
                Image support in IDataView and transforms.#489
                Assignees
                Ivanidzo4ka
                Labels
                APIIssues pertaining the friendly APIenhancementNew feature or request
                Milestone
                0718
                Description
                @Ivanidzo4ka
                Ivanidzo4ka
                opened on Jul 3, 2018
                Currently only way to work with images is by manually loading pixels arrays into array field in the class and wrap that class into collection data source. Which is not pleasant.
                I propose to add support for type Image in DataView type system which can be based on System.Drawing.Bitmap This way user can specify Bitmap image in their class and it will be properly consumed by CollectionDataSource.
                Also I suggest to implement transform which would accept column with file paths and load this images during pipeline execution. Which allow you to save memory in case of streaming dataview, some trainers support streaming structure of data, and this way you need to keep in memory only one image for current row instead of images for all rows.
                Since quite often you need to modify images (Re-scale them, transform image into Grayscale, etc) it make sense to keep Image format as "image" and not directly convert it into vector of floats which we require for learners. Which means we need transform which would convert Image type into feature vector.
                To summarize I propose to create following:
                Image type based on Bitmap.
                
ImageLoader transform
                
ImageResizer transform
                
ImageGrayscale transform
                
ImageToPixels transform
                @shauheen@TomFinley@glebuk
                Metadata
                Metadata
                Assignees
                Labels
                APIIssues pertaining the friendly APIenhancementNew feature or request
                Type
                No type
                Projects
                No projects
                  Milestone
                  Relationships
                  None yet
                  Development
                  No branches or pull requests
                  Issue actions