From fa7eb899c46dac78328a14bc38e4dd1870a26370 Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 05:48:24 +0300 Subject: [PATCH 01/26] Native operating system drag and drop, carrying the clipboard's own payload Codename One's drag and drop has always been lightweight: setDraggable and setDropTarget move a rendered image around inside one form. It never leaves the application, so it cannot drop a file on the desktop, cannot carry text into another application's window, and cannot receive anything from one. This adds the other half. The payload is a ClipboardContent -- the same object a copy publishes -- because a drag is a copy the user aims with the pointer: whatever the application can already put on the clipboard it can already drag out, and whatever it can paste it can already accept as a drop. Offering several representations is what lets one drag land correctly in unrelated applications; a text editor takes text/html, a plain text field takes text/plain, and the desktop takes the file list. Core ---- Label file = new Label("report.pdf"); file.setNativeDragOperation(NativeDragOperation.createFileDrag(paths)); inbox.setNativeDropTarget(true); inbox.addNativeDropListener(e -> ((NativeDropEvent)e).getFiles() ...); ClipboardContent gains lazily built representations. That is what makes dragging a file out workable: the drag has to name the file when it starts, but the user may drop it nowhere, so setDataProvider declares the representation without paying for it and the file is written at the moment a receiver reads it. It also gains setFiles/getFiles, which replaces the String-or-String[] duality every port was open-coding, and text/uri-list. NativeDragOperation carries the payload, the allowed actions and the drag image. ACTION_MOVE means the receiver takes ownership and the source deletes its copy; the source only learns whether that happened once the platform has finished, so the outcome arrives through a completion listener rather than from the call that started the drag. Threading. Drops arrive on the platform's own drag thread. The target is resolved there, from the accepted MIME types and actions alone, and the callbacks run on the event dispatch thread. That is not fastidiousness: in the JavaSE port the event dispatch thread blocks on the AWT thread to blit every frame, so an AWT callback that waits on the event dispatch thread deadlocks on the first drag. The consequence is that a MIME filter is exact from the first drag event while a decision made inside a callback reaches the cursor one event later, which is a frame. canAcceptNativeDrop is the one method that runs off the event dispatch thread, and says so. Ports ----- JavaSE (the simulator and "run as desktop app"): both directions, through AWT's own drag machinery, so a drag ends on another window, on the desktop or in a file manager. The transferable that publishes a copy now publishes a drag too; it derives its flavors from the MIME types alone rather than by reading values, which is what keeps a promised file unwritten until the drop. Android: startDragAndDrop with DRAG_FLAG_GLOBAL, so a drag crosses applications from Nougat onwards. The ClipData conversion the clipboard already had is now shared with the drag rather than duplicated, including the file provider URIs that let the receiving application read generated bytes. iOS, iPadOS and Mac Catalyst: UIDragInteraction and UIDropInteraction. UIKit owns the gesture -- its own recognizer decides a drag has begun and then asks what is being dragged -- so the framework stages the operation on the press and the native side announces the session afterwards. The payload is fetched at that later moment, so a drag offering a file the application has not written yet does not write it every time the user merely touches the component. Everything else answers false from NativeDragAndDrop.isSupported() and keeps the lightweight drag and drop unchanged. Not covered: the JavaScript port and the native macOS, Windows and Linux ports. Also fixed here, found while reviewing: a top level primes drag and drop twice per press -- once on the component under the pointer, once on its nearest draggable ancestor -- and the second pass discarded what the first had staged when the drag source sat between the two. Co-Authored-By: Claude Opus 5 (1M context) --- .../impl/CodenameOneImplementation.java | 86 +++ .../com/codename1/ui/ClipboardContent.java | 99 ++- .../codename1/ui/ClipboardDataProvider.java | 54 ++ .../src/com/codename1/ui/Component.java | 336 ++++++++++ CodenameOne/src/com/codename1/ui/Form.java | 9 + .../com/codename1/ui/NativeDragAndDrop.java | 622 ++++++++++++++++++ .../com/codename1/ui/NativeDragOperation.java | 265 ++++++++ .../src/com/codename1/ui/NativeDropEvent.java | 154 +++++ CodenameOne/src/com/codename1/ui/Window.java | 9 + .../com/codename1/ui/events/ActionEvent.java | 17 + .../impl/android/AndroidImplementation.java | 219 +++--- .../android/AndroidNativeDragAndDrop.java | 308 +++++++++ .../impl/javase/JavaSENativeDragAndDrop.java | 550 ++++++++++++++++ .../com/codename1/impl/javase/JavaSEPort.java | 170 ++++- Ports/iOSPort/nativeSources/CN1DragAndDrop.h | 102 +++ Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 521 +++++++++++++++ .../CodenameOne_GLViewController.m | 5 + Ports/iOSPort/nativeSources/IOSNative.m | 78 +++ .../codename1/impl/ios/IOSImplementation.java | 176 +++++ .../src/com/codename1/impl/ios/IOSNative.java | 48 ++ .../NativeDragAndDropSample.java | 224 +++++++ .../advancedtopics/NativeDragAndDropDemo.java | 130 ++++ .../Advanced-Topics-Under-The-Hood.asciidoc | 105 ++- .../TestCodenameOneImplementation.java | 72 ++ .../codename1/ui/NativeDragAndDropTest.java | 487 ++++++++++++++ .../javase/JavaSENativeDragAndDropTest.java | 256 +++++++ 26 files changed, 4988 insertions(+), 114 deletions(-) create mode 100644 CodenameOne/src/com/codename1/ui/ClipboardDataProvider.java create mode 100644 CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java create mode 100644 CodenameOne/src/com/codename1/ui/NativeDragOperation.java create mode 100644 CodenameOne/src/com/codename1/ui/NativeDropEvent.java create mode 100644 Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java create mode 100644 Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java create mode 100644 Ports/iOSPort/nativeSources/CN1DragAndDrop.h create mode 100644 Ports/iOSPort/nativeSources/CN1DragAndDrop.m create mode 100644 Samples/samples/NativeDragAndDropSample/NativeDragAndDropSample.java create mode 100644 docs/demos/common/src/main/java/com/codenameone/developerguide/advancedtopics/NativeDragAndDropDemo.java create mode 100644 maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java create mode 100644 maven/javase/src/test/java/com/codename1/impl/javase/JavaSENativeDragAndDropTest.java diff --git a/CodenameOne/src/com/codename1/impl/CodenameOneImplementation.java b/CodenameOne/src/com/codename1/impl/CodenameOneImplementation.java index 3db0273961c..56aeff8335c 100644 --- a/CodenameOne/src/com/codename1/impl/CodenameOneImplementation.java +++ b/CodenameOne/src/com/codename1/impl/CodenameOneImplementation.java @@ -80,6 +80,7 @@ import com.codename1.ui.CN; import com.codename1.ui.Command; import com.codename1.ui.ClipboardContent; +import com.codename1.ui.NativeDragOperation; import com.codename1.ui.Component; import com.codename1.ui.Container; import com.codename1.ui.Dialog; @@ -5566,6 +5567,91 @@ public void installNativeTheme() { throw new RuntimeException(); } + // ------------------------------------------------------------------------------------ + // Native (operating system) drag and drop. The payload is a ClipboardContent, the same one + // copyToClipboard publishes, because a drag is a copy the user aims with the pointer. + // + // A port implements the outbound half here and calls + // com.codename1.ui.NativeDragAndDrop for the inbound half. + // ------------------------------------------------------------------------------------ + + /// Returns true when this platform can hand a drag to the operating system. False here + /// leaves the framework with only its lightweight in-form drag and drop, which is what + /// every port had before. + /// + /// #### Returns + /// + /// true when native drag and drop is available + public boolean isNativeDragAndDropSupported() { + return false; + } + + /// Returns true when a drag started in this application can be dropped outside it -- on the + /// desktop, in a file manager, or in another application's window. Ports that can route a + /// drag between their own components but not out of the application return false while + /// still returning true from `#isNativeDragAndDropSupported()`. + /// + /// #### Returns + /// + /// true when a drag can leave the application + public boolean isNativeDragOutsideApplicationSupported() { + return isNativeDragAndDropSupported(); + } + + /// Hands the port the drag that would start if the press currently down turns into a drag. + /// + /// Ports whose platform owns the gesture -- where the operating system's own recognizer + /// decides a drag has begun and then asks what is being dragged -- answer from what they + /// were given here. Ports that start the session themselves can ignore this and use + /// `#startNativeDrag(com.codename1.ui.NativeDragOperation)`. + /// + /// Invoked on the event dispatch thread, once per press. A press that produces no drag is + /// followed by `#cancelNativeDrag()`. + /// + /// #### Parameters + /// + /// - `op`: the drag that is now possible + public void prepareNativeDrag(NativeDragOperation op) { + } + + /// Starts a native drag session now, because the pointer has moved far enough to be a drag. + /// Invoked on the event dispatch thread while the pointer is still down. + /// + /// A port that starts the session must eventually report the outcome through + /// `com.codename1.ui.NativeDragAndDrop#dragCompleted(int)`, or a source that offered a move + /// never learns whether to delete its copy. + /// + /// #### Parameters + /// + /// - `op`: what is being dragged + /// + /// #### Returns + /// + /// true when the operating system took the drag + public boolean startNativeDrag(NativeDragOperation op) { + return false; + } + + /// Discards whatever `#prepareNativeDrag(com.codename1.ui.NativeDragOperation)` staged, + /// because the press turned out to be a click. + public void cancelNativeDrag() { + } + + /// True when the port needs the drag image at the moment the operation is staged rather + /// than when the drag begins. + /// + /// A port that starts the session itself renders the image then, which costs nothing for a + /// press that turns out to be a click. A port whose platform owns the drag gesture is asked + /// for the preview from inside that platform's own callback, which is not a moment at which + /// a component can be rendered, so it has to have the image already. + /// + /// #### Returns + /// + /// true to render the drag image on every press over a native drag source + public boolean isNativeDragImageNeededOnPrepare() { + return false; + } + /// Performs a clipboard copy operation, if the native clipboard is supported by the implementation it would be used /// /// #### Parameters diff --git a/CodenameOne/src/com/codename1/ui/ClipboardContent.java b/CodenameOne/src/com/codename1/ui/ClipboardContent.java index c26f5a58092..cad251c2884 100644 --- a/CodenameOne/src/com/codename1/ui/ClipboardContent.java +++ b/CodenameOne/src/com/codename1/ui/ClipboardContent.java @@ -46,12 +46,40 @@ public class ClipboardContent { public static final String MIME_GIF = "image/gif"; /// A local file reference (a file path / URI `String`, or a `String[]` for several files). public static final String MIME_FILE = "application/x-file-list"; + /// A newline separated list of URIs, the format desktop browsers and file managers use when + /// a link or a file is dragged out of them. + public static final String MIME_URI_LIST = "text/uri-list"; private final List mimeTypes = new ArrayList(); private final List values = new ArrayList(); /// Adds or replaces a representation. Passing null removes the MIME type. public ClipboardContent setData(String mimeType, Object value) { + return put(mimeType, value); + } + + /// Declares a representation that is only built if something reads it, which is how a drag + /// offers a file it has not written yet or an image it has not encoded yet. + /// + /// The provider is asked at most once and its answer is then cached, so + /// `#getData(java.lang.String)` behaves exactly as it would for a value passed to + /// `#setData(java.lang.String, java.lang.Object)`. It may run on a native clipboard or drag + /// thread; see `ClipboardDataProvider`. Passing a null provider removes the MIME type. + /// + /// #### Parameters + /// + /// - `mimeType`: the MIME type this provider can produce + /// + /// - `provider`: the provider, or null to remove the representation + /// + /// #### Returns + /// + /// this instance, for chaining + public ClipboardContent setDataProvider(String mimeType, ClipboardDataProvider provider) { + return put(mimeType, provider == null ? null : new LazyValue(provider)); + } + + private ClipboardContent put(String mimeType, Object value) { String normalized = normalizeMimeType(mimeType); if (normalized.length() == 0) { throw new IllegalArgumentException("MIME type must not be empty"); @@ -74,9 +102,41 @@ public ClipboardContent setData(String mimeType, Object value) { } /// Returns the representation for a MIME type, or null when it isn't available. + /// + /// A representation registered through + /// `#setDataProvider(java.lang.String, com.codename1.ui.ClipboardDataProvider)` is produced + /// here, on the first call for that MIME type. public Object getData(String mimeType) { int index = mimeTypes.indexOf(normalizeMimeType(mimeType)); - return index < 0 ? null : values.get(index); + if (index < 0) { + return null; + } + Object value = values.get(index); + if (value instanceof LazyValue) { + return ((LazyValue) value).resolve(mimeTypes.get(index)); + } + return value; + } + + /// A provider plus the value it produced. The value is resolved once and remembered, so a + /// target that asks twice -- as a drop does when it queries and then reads -- does not run + /// the provider twice and does not, for instance, write the promised file twice. + private static final class LazyValue { + private final ClipboardDataProvider provider; + private Object resolved; + private boolean done; + + LazyValue(ClipboardDataProvider provider) { + this.provider = provider; + } + + synchronized Object resolve(String mimeType) { + if (!done) { + done = true; + resolved = provider.getClipboardData(mimeType); + } + return resolved; + } } /// Returns the binary (`byte[]`) representation for a MIME type -- e.g. the raw bytes of an image @@ -102,6 +162,43 @@ public String[] getMimeTypes() { return mimeTypes.toArray(new String[mimeTypes.size()]); } + /// Sets the `#MIME_FILE` representation from a list of file paths or `file:` URIs. + /// + /// Ports differ on whether a single file is carried as a `String` or a one element + /// `String[]`; this writes the form the rest of the framework expects, and + /// `#getFiles()` reads either. + /// + /// #### Parameters + /// + /// - `paths`: the file paths, or null to remove the representation + /// + /// #### Returns + /// + /// this instance, for chaining + public ClipboardContent setFiles(String[] paths) { + if (paths == null || paths.length == 0) { + return setData(MIME_FILE, null); + } + if (paths.length == 1) { + return setData(MIME_FILE, paths[0]); + } + return setData(MIME_FILE, paths.clone()); + } + + /// Returns the `#MIME_FILE` representation as a path array regardless of whether the + /// producer stored one `String` or a `String[]`, or null when no files are available. + public String[] getFiles() { + Object value = getData(MIME_FILE); + if (value instanceof String[]) { + String[] paths = (String[]) value; + return paths.length == 0 ? null : paths.clone(); + } + if (value instanceof String && ((String) value).length() > 0) { + return new String[] { (String) value }; + } + return null; + } + /// Returns the first available MIME type from the caller's preference list, or null. public String findPreferredMimeType(String[] preferredMimeTypes) { if (preferredMimeTypes == null) { diff --git a/CodenameOne/src/com/codename1/ui/ClipboardDataProvider.java b/CodenameOne/src/com/codename1/ui/ClipboardDataProvider.java new file mode 100644 index 00000000000..0d6d528f213 --- /dev/null +++ b/CodenameOne/src/com/codename1/ui/ClipboardDataProvider.java @@ -0,0 +1,54 @@ +/* + * Copyright (c) 2026, Codename One and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. Codename One designates this + * particular file as subject to the "Classpath" exception as provided + * by Oracle in the LICENSE file that accompanied this code. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Codename One through http://www.codenameone.com/ if you + * need additional information or have any questions. + */ + + +package com.codename1.ui; + +/// Produces a clipboard or drag-and-drop representation on demand, so a payload that is +/// expensive to build is only built if something actually asks for it. +/// +/// This is what makes "drag a file out of the application" workable. A drag that offers +/// `ClipboardContent#MIME_FILE` has to name the file when the drag *starts*, but the drop may +/// never happen -- the user may let go over nothing -- and the target may prefer a different +/// representation entirely. Registering a provider with +/// `ClipboardContent#setDataProvider(java.lang.String, com.codename1.ui.ClipboardDataProvider)` +/// declares that the representation is available without paying for it up front; the bytes are +/// written, or the temporary file created, at the moment the receiving application reads that +/// MIME type. +/// +/// A provider is invoked at most once per `ClipboardContent` and MIME type -- the result is +/// cached -- and it may be invoked from a native drag or clipboard thread rather than the event +/// dispatch thread, so it must not touch the user interface. +public interface ClipboardDataProvider { + /// Produces the value for one representation. + /// + /// #### Parameters + /// + /// - `mimeType`: the MIME type being requested, always one this provider was registered for + /// + /// #### Returns + /// + /// the value, normally a `String`, a `String[]` of file paths or a `byte[]`, or null when + /// the representation turned out to be unavailable + public Object getClipboardData(String mimeType); +} diff --git a/CodenameOne/src/com/codename1/ui/Component.java b/CodenameOne/src/com/codename1/ui/Component.java index d059fa7655a..9bc4437239f 100644 --- a/CodenameOne/src/com/codename1/ui/Component.java +++ b/CodenameOne/src/com/codename1/ui/Component.java @@ -411,6 +411,17 @@ static int rubberBandDecompress(int compressed, int dim) { private boolean draggable; private boolean dragAndDropInitialized; private boolean dropTarget; + /// Native (operating system) drag and drop state. Kept beside the lightweight drag and drop + /// fields above because the two are alternatives for the same gesture: a component that is a + /// native drag source hands the press to the platform, and the lightweight drag never runs. + private boolean nativeDragSource; + private boolean nativeDropTarget; + private NativeDragOperation nativeDragOperation; + private String[] acceptedDropMimeTypes; + private int acceptedDropActions = NativeDragOperation.ACTION_COPY + | NativeDragOperation.ACTION_MOVE | NativeDragOperation.ACTION_LINK; + private EventDispatcher nativeDropListeners; + private EventDispatcher nativeDragOverListeners; private Image dragImage; private Component dropTargetComponent; private int dragCallbacks = 0; @@ -5965,6 +5976,28 @@ public void pointerPressed(int x, int y) { void initDragAndDrop(int x, int y) { Component leadParent = LeadUtil.leadParentImpl(this); leadParent.dragAndDropInitialized = leadParent.isDragAndDropOperation(x, y); + // Native drag and drop is primed from the same place, so a native drag source is + // pressed, dragged and released through exactly the gesture a draggable component is. + NativeDragAndDrop.pressedOn(leadParent, x, y); + } + + /// Abandons a lightweight drag that has already started, without running the drop + /// machinery. Used when a native drag takes the gesture over: the port stops delivering + /// pointer drags at that point, so the lightweight drag would otherwise stay activated with + /// its image stranded where the gesture began. + void cancelLightweightDrag() { + Component leadParent = LeadUtil.leadParentImpl(this); + if (leadParent.dragActivated) { + Form p = getComponentForm(); + if (p != null) { + p.setDraggedComponent(null); + p.repaint(); + } + } + leadParent.dragActivated = false; + leadParent.dragAndDropInitialized = false; + leadParent.dragImage = null; + leadParent.dropTargetComponent = null; } /// If this Component is focused, the pointer released event @@ -6450,6 +6483,309 @@ public void removeDragOverListener(ActionListener l) { } } + // ------------------------------------------------------------------------------------ + // Native (operating system) drag and drop. + // + // The listeners and callbacks above move a component around inside one form. The ones + // below hand the drag to the platform, so it can end on the desktop, in a file manager or + // in another application -- and so a drag from any of those can end here. The payload is a + // ClipboardContent either way, which is the point: whatever the component can already copy + // it can already drag, and whatever it can already paste it can already accept. + // + // See NativeDragAndDrop for the platform support matrix and for starting a drag by hand. + // ------------------------------------------------------------------------------------ + + /// Returns true when a drag starting on this component is handed to the operating system. + public boolean isNativeDragSource() { + return nativeDragSource; + } + + /// Makes a drag that starts on this component an operating system drag, which can be + /// dropped outside the application. + /// + /// Supply what is being dragged either by calling + /// `#setNativeDragOperation(com.codename1.ui.NativeDragOperation)`, or by overriding + /// `#createNativeDragOperation(int, int)` when the payload depends on where the press + /// landed -- which item of a list was grabbed, for instance. + /// + /// This is independent of `#setDraggable(boolean)`. Where the platform has no native drag + /// and drop this flag simply does nothing, so a component that should be draggable either + /// way sets both and the native session, when there is one, takes precedence. + /// + /// #### Parameters + /// + /// - `nativeDragSource`: true to hand drags on this component to the operating system + public void setNativeDragSource(boolean nativeDragSource) { + this.nativeDragSource = nativeDragSource; + } + + /// Returns the operation this component drags, or null when it supplies one per press by + /// overriding `#createNativeDragOperation(int, int)`. + public NativeDragOperation getNativeDragOperation() { + return nativeDragOperation; + } + + /// Sets what dragging this component puts on the operating system's drag, and makes the + /// component a native drag source. Passing null clears both. + /// + /// The operation is reusable: the same instance is offered for every drag of this + /// component, so it must not hold state from a previous session. Anything expensive in the + /// payload belongs behind + /// `ClipboardContent#setDataProvider(java.lang.String, com.codename1.ui.ClipboardDataProvider)`. + /// + /// #### Parameters + /// + /// - `nativeDragOperation`: what to drag, or null to stop being a drag source + public void setNativeDragOperation(NativeDragOperation nativeDragOperation) { + this.nativeDragOperation = nativeDragOperation; + this.nativeDragSource = nativeDragOperation != null; + } + + /// Produces the operation for a drag starting at the given position, invoked on the event + /// dispatch thread as the press is dispatched. Override when the payload depends on where + /// the user grabbed the component; the default returns whatever + /// `#setNativeDragOperation(com.codename1.ui.NativeDragOperation)` was given. + /// + /// Returning null, or an operation allowing no actions, leaves the gesture alone -- which + /// is how a component refuses to be dragged from a particular spot. + /// + /// #### Parameters + /// + /// - `x`: the absolute x position of the press + /// + /// - `y`: the absolute y position of the press + /// + /// #### Returns + /// + /// the drag to start, or null for none + protected NativeDragOperation createNativeDragOperation(int x, int y) { + return nativeDragOperation; + } + + /// Returns true when this component accepts drops coming from the operating system. + public boolean isNativeDropTarget() { + return nativeDropTarget; + } + + /// Lets this component receive drops from the operating system: from another application, + /// from a file manager, from the desktop, or from elsewhere in this application. + /// + /// The deepest component under the pointer that is a native drop target and accepts the + /// content wins, so a target nested inside another target takes precedence -- and a target + /// that refuses a particular payload lets an ancestor have it. + /// + /// This is independent of `#setDropTarget(boolean)`, which governs the lightweight drag and + /// drop inside the form. + /// + /// #### Parameters + /// + /// - `nativeDropTarget`: true to accept operating system drops + public void setNativeDropTarget(boolean nativeDropTarget) { + this.nativeDropTarget = nativeDropTarget; + } + + /// Returns the MIME types this component accepts, or null when it accepts anything. + public String[] getAcceptedDropMimeTypes() { + return acceptedDropMimeTypes == null ? null : acceptedDropMimeTypes.clone(); + } + + /// Restricts the drops this component accepts to drags offering at least one of these MIME + /// types, so a drag carrying anything else passes through to whatever is behind it. + /// + /// This filter is consulted on the platform's drag thread rather than the event dispatch + /// thread, which is what lets the answer be exact from the very first drag event; see + /// `NativeDragAndDrop#dragOver(int, int, int, com.codename1.ui.ClipboardContent, int)`. + /// + /// #### Parameters + /// + /// - `acceptedDropMimeTypes`: the MIME types, for instance + /// `ClipboardContent#MIME_FILE`, or null to accept anything + public void setAcceptedDropMimeTypes(String... acceptedDropMimeTypes) { + this.acceptedDropMimeTypes = acceptedDropMimeTypes == null || acceptedDropMimeTypes.length == 0 + ? null : acceptedDropMimeTypes.clone(); + } + + /// Returns the bit set of actions this component is willing to perform on a drop. + public int getAcceptedDropActions() { + return acceptedDropActions; + } + + /// Restricts what this component will do with a drop -- a target that can only copy should + /// not be offered a move, because the source deletes its data when a move completes. + /// + /// #### Parameters + /// + /// - `acceptedDropActions`: any combination of `NativeDragOperation#ACTION_COPY`, + /// `NativeDragOperation#ACTION_MOVE` and `NativeDragOperation#ACTION_LINK` + public void setAcceptedDropActions(int acceptedDropActions) { + this.acceptedDropActions = acceptedDropActions; + } + + /// Decides whether this component wants a drag carrying these representations at all. + /// + /// #### Threading + /// + /// Invoked on the platform's drag thread, not the event dispatch thread, because the + /// operating system needs the answer while the pointer is moving. Read the content and + /// this component's own configuration; do not touch the user interface, start animations + /// or block. Everything that needs the event dispatch thread belongs in + /// `#nativeDragEnter(com.codename1.ui.NativeDropEvent)` and its siblings. + /// + /// The default accepts anything unless + /// `#setAcceptedDropMimeTypes(java.lang.String...)` narrowed it. + /// + /// #### Parameters + /// + /// - `content`: the representations the drag is offering + /// + /// #### Returns + /// + /// true to be considered as the target + protected boolean canAcceptNativeDrop(ClipboardContent content) { + if (acceptedDropMimeTypes == null) { + return true; + } + if (content == null) { + return false; + } + return content.findPreferredMimeType(acceptedDropMimeTypes) != null; + } + + /// Callback invoked on the event dispatch thread when a native drag enters this component. + /// Use it to highlight the drop location, and `NativeDropEvent#accept(int)` or + /// `NativeDropEvent#reject()` to change what the cursor tells the user. + /// + /// #### Parameters + /// + /// - `ev`: the drag + protected void nativeDragEnter(NativeDropEvent ev) { + } + + /// Callback invoked on the event dispatch thread as a native drag moves over this + /// component. Delivered at most once at a time -- a new one is only queued after the + /// previous returned -- so a slow callback throttles itself instead of flooding the event + /// dispatch thread. + /// + /// #### Parameters + /// + /// - `ev`: the drag + protected void nativeDragOver(NativeDropEvent ev) { + } + + /// Callback invoked on the event dispatch thread when a native drag leaves this component + /// without dropping. Clear whatever `#nativeDragEnter(com.codename1.ui.NativeDropEvent)` + /// highlighted. The event carries no content. + /// + /// #### Parameters + /// + /// - `ev`: the drag + protected void nativeDragExit(NativeDropEvent ev) { + } + + /// Callback invoked on the event dispatch thread when a native drag is dropped on this + /// component. `NativeDropEvent#getContent()` is fully materialized by this point, so the + /// bytes and file paths can be read here or kept for later. + /// + /// #### Parameters + /// + /// - `ev`: the drop + protected void nativeDrop(NativeDropEvent ev) { + } + + /// Adds a listener invoked on the event dispatch thread when a native drag is dropped on + /// this component. The event is a `NativeDropEvent`. + /// + /// #### Parameters + /// + /// - `l`: the listener + public void addNativeDropListener(ActionListener l) { + if (nativeDropListeners == null) { + nativeDropListeners = new EventDispatcher(); + } + nativeDropListeners.addListener(l); + } + + /// Removes a listener added by + /// `#addNativeDropListener(com.codename1.ui.events.ActionListener)`. + /// + /// #### Parameters + /// + /// - `l`: the listener + public void removeNativeDropListener(ActionListener l) { + if (nativeDropListeners != null) { + nativeDropListeners.removeListener(l); + if (!nativeDropListeners.hasListeners()) { + nativeDropListeners = null; + } + } + } + + /// Adds a listener invoked on the event dispatch thread as a native drag enters, moves over + /// and leaves this component. The event is a `NativeDropEvent`; its + /// `com.codename1.ui.events.ActionEvent#getEventType()` says which of the three it is. + /// + /// #### Parameters + /// + /// - `l`: the listener + public void addNativeDragOverListener(ActionListener l) { + if (nativeDragOverListeners == null) { + nativeDragOverListeners = new EventDispatcher(); + } + nativeDragOverListeners.addListener(l); + } + + /// Removes a listener added by + /// `#addNativeDragOverListener(com.codename1.ui.events.ActionListener)`. + /// + /// #### Parameters + /// + /// - `l`: the listener + public void removeNativeDragOverListener(ActionListener l) { + if (nativeDragOverListeners != null) { + nativeDragOverListeners.removeListener(l); + if (!nativeDragOverListeners.hasListeners()) { + nativeDragOverListeners = null; + } + } + } + + /// Routes one native drag callback to the override and then to the listeners. Invoked on + /// the event dispatch thread by `NativeDragAndDrop`. + /// + /// #### Parameters + /// + /// - `ev`: the event + void dispatchNativeDropEvent(NativeDropEvent ev) { + switch (ev.getEventType()) { + case NativeDragEnter: + nativeDragEnter(ev); + fireNativeDragOver(ev); + break; + case NativeDragOver: + nativeDragOver(ev); + fireNativeDragOver(ev); + break; + case NativeDragExit: + nativeDragExit(ev); + fireNativeDragOver(ev); + break; + case NativeDrop: + nativeDrop(ev); + if (nativeDropListeners != null && nativeDropListeners.hasListeners()) { + nativeDropListeners.fireActionEvent(ev); + } + break; + default: + break; + } + } + + private void fireNativeDragOver(NativeDropEvent ev) { + if (nativeDragOverListeners != null && nativeDragOverListeners.hasListeners()) { + nativeDragOverListeners.fireActionEvent(ev); + } + } + /// Callback indicating that the drag has finished either via drop or by releasing the component /// /// #### Parameters diff --git a/CodenameOne/src/com/codename1/ui/Form.java b/CodenameOne/src/com/codename1/ui/Form.java index 81c67eefc02..65a3b29cb92 100644 --- a/CodenameOne/src/com/codename1/ui/Form.java +++ b/CodenameOne/src/com/codename1/ui/Form.java @@ -4256,6 +4256,12 @@ public void pointerDragged(int x, int y) { stylusCmp.fireStylusEvent(ActionEvent.Type.PointerDrag, x, y); } } + // A press that landed on a native drag source becomes an operating system drag here, + // as soon as it has moved far enough to be a drag rather than a click. From that point + // the platform owns the gesture, so nothing below runs for it. + if (NativeDragAndDrop.pointerDragged(x, y)) { + return; + } // disable the drag stop flag if we are dragging again boolean isScrollWheeling = Display.impl.isScrollWheeling(); if (dragStopFlag) { @@ -4582,6 +4588,9 @@ public void pointerReleased(int x, int y) { stylusCmp.fireStylusEvent(ActionEvent.Type.PointerReleased, x, y); } } + // A press that never became a drag releases the operation the press staged, so a + // later gesture somewhere else cannot start the drag this one declined to. + NativeDragAndDrop.pointerReleased(); try { Component origPressedCmp = pressedCmp; setRippleMotion(null); diff --git a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java new file mode 100644 index 00000000000..b31a400a019 --- /dev/null +++ b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java @@ -0,0 +1,622 @@ +/* + * Copyright (c) 2026, Codename One and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. Codename One designates this + * particular file as subject to the "Classpath" exception as provided + * by Oracle in the LICENSE file that accompanied this code. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Codename One through http://www.codenameone.com/ if you + * need additional information or have any questions. + */ + + +package com.codename1.ui; + +import com.codename1.io.Log; +import com.codename1.ui.events.ActionEvent; + +/// Drag and drop through the operating system rather than inside the application. +/// +/// Codename One has always had a lightweight drag and drop -- `Component#setDraggable(boolean)` +/// and `Component#setDropTarget(boolean)` -- which moves a rendered image around inside one +/// form. That never leaves the application, so it cannot drop a file on the desktop, cannot +/// carry text into another application's window, and cannot receive anything from one. +/// +/// This class is the other half: it hands the drag to the operating system's own drag machinery, +/// using the same `ClipboardContent` a copy publishes as the payload. That is the whole idea -- +/// a drag is a copy that the user aims with the pointer, so anything the application can already +/// put on the clipboard it can already drag out, and anything it can paste it can already accept +/// as a drop. +/// +/// #### Dragging out +/// +/// ```java +/// Label file = new Label("report.pdf"); +/// file.setNativeDragOperation(NativeDragOperation.createFileDrag( +/// new String[] { FileSystemStorage.getInstance().getAppHomePath() + "report.pdf" })); +/// ``` +/// +/// Dropping that on the desktop, on a mail composer or into a file manager copies the file, +/// because the receiving application asked for `ClipboardContent#MIME_FILE` and the drag +/// offered it. Offer several representations and every receiver takes the best one it +/// understands. +/// +/// #### Receiving a drop +/// +/// ```java +/// Container inbox = new Container(); +/// inbox.setNativeDropTarget(true); +/// inbox.addNativeDropListener(e -> { +/// NativeDropEvent drop = (NativeDropEvent)e; +/// String[] files = drop.getFiles(); +/// ... +/// }); +/// ``` +/// +/// #### Where it works +/// +/// Native drag and drop needs the platform to have it. Check `#isSupported()` before offering +/// the affordance, and `#isDragOutsideApplicationSupported()` before promising the user that a +/// drag can leave the application: a desktop can drop onto any other window, a tablet can drop +/// into another application beside it, and a phone in full screen has nowhere for a drag to go +/// even though drags within the application still work. Where nothing is supported the calls +/// here are harmless no-ops and the lightweight drag and drop is unaffected. +public final class NativeDragAndDrop { + /// A press further than this from where it started is a drag rather than a click. Measured + /// in millimetres so it is a finger on a phone and a pointer on a desktop. + private static final float DRAG_THRESHOLD_MM = 1.5f; + + /// The operation prepared by the press that is currently down, waiting to see whether the + /// user drags. + /// + /// Written on the event dispatch thread. Read from a native drag thread as well, because a + /// platform that owns the drag gesture itself -- iOS and iPadOS do -- announces the session + /// it started through `#dragSessionStarted()` and this is the operation it started. + private static volatile NativeDragOperation pending; + private static volatile Component pendingSource; + private static int pressX; + private static int pressY; + /// Set once this press has been offered to the port, so a platform that declined to start + /// the session is not asked again on every drag event of the same gesture. + private static boolean startOffered; + + /// The session the operating system is currently running, or null. Written on the event + /// dispatch thread and read from the native drag thread, hence volatile. + private static volatile NativeDragOperation active; + + /// The drop target the drag is currently over, and the action it last agreed to. Both are + /// read and written from the native drag thread; see the note on + /// `#dragOver(int, int, int, com.codename1.ui.ClipboardContent, int)` about why the answer + /// given to the operating system is the previous callback's. + private static volatile Component currentTarget; + private static volatile int currentAction = NativeDragOperation.ACTION_NONE; + private static volatile boolean overDispatchPending; + + private NativeDragAndDrop() { + } + + /// Returns true when this platform can drag and drop through the operating system at all. + /// Where this is false every method here does nothing and reports failure, so no call site + /// needs to be conditional -- but an application that shows a "drag me" affordance should + /// hide it. + public static boolean isSupported() { + return Display.impl != null && Display.impl.isNativeDragAndDropSupported(); + } + + /// Returns true when a drag started here can be dropped outside the application: on the + /// desktop, in a file manager or in another application's window. + /// + /// This is narrower than `#isSupported()`. A platform can route drags between components, + /// and between this application's own windows, while still refusing to let one leave -- + /// which is the normal state of affairs on a phone. + public static boolean isDragOutsideApplicationSupported() { + return Display.impl != null && Display.impl.isNativeDragOutsideApplicationSupported(); + } + + /// Starts a native drag immediately, for an application that decides on its own that a drag + /// has begun -- from a long press, or a menu item -- rather than letting a component do it + /// through `Component#setNativeDragSource(boolean)`. + /// + /// Call this on the event dispatch thread while the pointer is still down; a drag the user + /// is not currently holding cannot be aimed and platforms reject it. + /// + /// #### Parameters + /// + /// - `source`: the component the drag comes from, used for the default drag image and + /// reported by `NativeDragOperation#getSource()`. May be null. + /// + /// - `op`: what is being dragged + /// + /// #### Returns + /// + /// true when the operating system took the drag; false when the platform has no native drag + /// and drop, or refused to start a session + public static boolean startDrag(Component source, NativeDragOperation op) { + if (op == null || !isSupported()) { + return false; + } + op.setSource(source); + active = op; + currentTarget = null; + currentAction = NativeDragOperation.ACTION_NONE; + boolean started = false; + try { + started = Display.impl.startNativeDrag(op); + } catch (Throwable err) { + // A port that cannot start a session must not take the application down with it; + // the gesture simply stays a lightweight one. + Log.e(err); + } + if (!started) { + active = null; + } + return started; + } + + /// Reports that the platform started a drag session on its own, for the operation the press + /// prepared. Ports whose operating system owns the drag gesture -- where a long press, not + /// the framework's own threshold, is what begins a drag -- call this instead of returning + /// true from `com.codename1.impl.CodenameOneImplementation#startNativeDrag(com.codename1.ui.NativeDragOperation)`. + /// + /// #### Returns + /// + /// the operation the session is carrying, or null when nothing was prepared -- in which case + /// the port should refuse to start a session + public static NativeDragOperation dragSessionStarted() { + NativeDragOperation op = pending; + if (op == null) { + return null; + } + final Component source = pendingSource; + pending = null; + pendingSource = null; + active = op; + currentTarget = null; + currentAction = NativeDragOperation.ACTION_NONE; + if (source != null) { + // On the event dispatch thread, because it repaints. A component that is draggable + // as well as a native drag source would otherwise be left mid-drag with its image + // stranded, since the platform stops delivering pointer drags once it takes over. + Display.getInstance().callSerially(new Runnable() { + public void run() { + source.cancelLightweightDrag(); + } + }); + } + return op; + } + + /// Returns the drag this application is currently running through the operating system, or + /// null when it is not dragging. A drop target uses this to tell a drag it started itself + /// from one that arrived from elsewhere, which `NativeDropEvent#isLocal()` reports. + public static NativeDragOperation getActiveDrag() { + return active; + } + + // ------------------------------------------------------------------------------------ + // Gesture plumbing. Called by the top level containers as a press is dispatched, so that a + // native drag source behaves exactly like a lightweight draggable one from the user's side. + // ------------------------------------------------------------------------------------ + + /// Prepares a drag for the press that just landed, so the port has the payload in hand + /// before the operating system's own gesture recognizer asks for it. Every press either + /// installs a new pending operation or clears the previous one, which is what keeps a drag + /// source that was pressed and released from being dragged by a later gesture somewhere + /// else. + static void pressedOn(Component cmp, int x, int y) { + if (pending != null && x == pressX && y == pressY) { + // The same press, dispatched a second time. A top level primes drag and drop on the + // component under the pointer and then again on its nearest draggable ancestor, and + // the ancestor walk below would not find a drag source that sits *between* the two + // -- so clearing here would throw away what the first call correctly staged. Every + // release clears the pending operation, so a later press cannot land on a stale one + // even at the very same pixel. + return; + } + pending = null; + pendingSource = null; + startOffered = false; + if (cmp == null || !isSupported()) { + return; + } + Component source = cmp; + while (source != null && !source.isNativeDragSource()) { + source = source.getParent(); + } + if (source == null) { + return; + } + NativeDragOperation op; + try { + op = source.createNativeDragOperation(x, y); + } catch (Throwable err) { + Log.e(err); + return; + } + if (op == null || op.getAllowedActions() == NativeDragOperation.ACTION_NONE) { + return; + } + op.setSource(source); + pending = op; + pendingSource = source; + pressX = x; + pressY = y; + try { + if (op.getDragImage() == null && Display.impl.isNativeDragImageNeededOnPrepare()) { + // The platform asks for the preview from inside its own gesture callback, which + // is not a moment at which a component can be rendered. Rendering here costs a + // snapshot per press on a drag source, which is what the lightweight drag has + // always cost when one starts. + op.setDragImage(source.getDragImage()); + op.setDragImageOffset(x - source.getAbsoluteX(), y - source.getAbsoluteY()); + } + Display.impl.prepareNativeDrag(op); + } catch (Throwable err) { + Log.e(err); + } + } + + /// Hands the gesture to the operating system once the pointer has moved far enough to be a + /// drag rather than a click. + /// + /// #### Parameters + /// + /// - `x`: the pointer position + /// + /// - `y`: the pointer position + /// + /// #### Returns + /// + /// true when the native drag has taken the gesture over and the framework should not also + /// treat it as a scroll or a lightweight drag + static boolean pointerDragged(int x, int y) { + NativeDragOperation op = pending; + if (op == null) { + // A session already running owns the gesture. Ports differ on whether they keep + // delivering pointer drags during a native drag; swallowing them here means the + // ones that do cannot scroll the surface out from under the drag. + return active != null; + } + int threshold = dragThreshold(); + if (Math.abs(x - pressX) < threshold && Math.abs(y - pressY) < threshold) { + return false; + } + if (startOffered) { + // Already offered for this gesture and not taken, which is what a platform that + // starts the session on its own recognizer looks like. Leave the gesture alone + // until that recognizer fires; it announces itself through dragSessionStarted(). + return active != null; + } + startOffered = true; + Component source = pendingSource; + if (op.getDragImage() == null && source != null) { + try { + op.setDragImage(source.getDragImage()); + // Only when the image is the one we just rendered from the component. An + // application that supplied its own image may also have positioned it, and + // overwriting that offset would tear the image away from the pointer. + op.setDragImageOffset(pressX - source.getAbsoluteX(), pressY - source.getAbsoluteY()); + } catch (Throwable err) { + Log.e(err); + } + } + if (!startDrag(source, op)) { + // The port did not take it. Keep the prepared operation: on a platform whose own + // gesture recognizer owns dragging, the session begins later and this is what it + // will carry. Where there is no native drag and drop at all nothing was prepared + // in the first place, so there is nothing to keep. + return false; + } + pending = null; + pendingSource = null; + if (source != null) { + // A component can be both draggable and a native drag source. The native session + // owns the gesture from here, and the port stops delivering pointer drags, so the + // lightweight drag would otherwise be left activated with its image stranded where + // the drag began. + source.cancelLightweightDrag(); + } + return true; + } + + /// Drops the operation prepared by a press that turned out to be a click. Called as the + /// pointer is released. + static void pointerReleased() { + startOffered = false; + if (pending != null) { + pending = null; + pendingSource = null; + try { + Display.impl.cancelNativeDrag(); + } catch (Throwable err) { + Log.e(err); + } + } + } + + private static int dragThreshold() { + try { + return Math.max(4, Display.getInstance().convertToPixels(DRAG_THRESHOLD_MM)); + } catch (Throwable err) { + return 8; + } + } + + // ------------------------------------------------------------------------------------ + // The receiving side. Ports call these from whatever thread the operating system hands + // them, which is not the event dispatch thread. + // ------------------------------------------------------------------------------------ + + /// Reports that a native drag has entered one of the application's surfaces. + /// + /// #### Parameters + /// + /// - `windowId`: the id of the window the drag is over, or zero for the main surface + /// + /// - `x`: the pointer position within that surface + /// + /// - `y`: the pointer position within that surface + /// + /// - `content`: the representations the drag is offering + /// + /// - `allowedActions`: the actions the source permits + /// + /// #### Returns + /// + /// the action a drop would perform right now, or `NativeDragOperation#ACTION_NONE` when + /// nothing under the pointer will take it + public static int dragEnter(int windowId, int x, int y, ClipboardContent content, int allowedActions) { + return dragOver(windowId, x, y, content, allowedActions); + } + + /// Reports that a native drag has moved over one of the application's surfaces, and answers + /// whether it would be accepted here. + /// + /// #### Threading + /// + /// The operating system needs the answer synchronously, while the framework's callbacks + /// have to run on the event dispatch thread -- and blocking a native drag thread on the + /// event dispatch thread deadlocks, because on some ports the event dispatch thread is + /// itself waiting on that native thread to paint. So the target is resolved here, on the + /// calling thread, from state that does not change under it, while + /// `Component#nativeDragOver(com.codename1.ui.NativeDropEvent)` and the listeners are + /// dispatched asynchronously; the value returned is the one *they* produced for the + /// previous event on this same target. A target that changes its mind therefore shows the + /// user the new cursor one drag event late, which is a frame, and never blocks. + /// + /// A target that refuses a drop outright should say so through + /// `Component#canAcceptNativeDrop(com.codename1.ui.ClipboardContent)` or the accepted MIME + /// list instead, both of which are consulted here and are therefore exact from the first + /// event. + /// + /// #### Parameters + /// + /// - `windowId`: the id of the window the drag is over, or zero for the main surface + /// + /// - `x`: the pointer position within that surface + /// + /// - `y`: the pointer position within that surface + /// + /// - `content`: the representations the drag is offering + /// + /// - `allowedActions`: the actions the source permits + /// + /// #### Returns + /// + /// the action a drop would perform right now, or `NativeDragOperation#ACTION_NONE` + public static int dragOver(final int windowId, final int x, final int y, + final ClipboardContent content, final int allowedActions) { + Component target = findTarget(windowId, x, y, content); + Component previous = currentTarget; + if (previous != target) { // NOPMD CompareObjectsWithEquals + currentTarget = target; + currentAction = target == null ? NativeDragOperation.ACTION_NONE + : (allowedActions & target.getAcceptedDropActions()) == 0 + ? NativeDragOperation.ACTION_NONE + : preferredAction(allowedActions & target.getAcceptedDropActions()); + dispatch(previous, ActionEvent.Type.NativeDragExit, content, x, y, allowedActions); + dispatch(target, ActionEvent.Type.NativeDragEnter, content, x, y, allowedActions); + return currentAction; + } + if (target == null) { + return NativeDragOperation.ACTION_NONE; + } + if (!overDispatchPending) { + overDispatchPending = true; + dispatch(target, ActionEvent.Type.NativeDragOver, content, x, y, allowedActions); + } + return currentAction; + } + + /// Reports that a native drag has left the application's surfaces without dropping. + /// + /// #### Parameters + /// + /// - `windowId`: the id of the window the drag left, or zero for the main surface + public static void dragExit(int windowId) { + Component previous = currentTarget; + currentTarget = null; + currentAction = NativeDragOperation.ACTION_NONE; + dispatch(previous, ActionEvent.Type.NativeDragExit, null, 0, 0, NativeDragOperation.ACTION_NONE); + } + + /// Delivers a native drop. + /// + /// The content must be fully materialized before this is called: on most platforms the + /// native transfer object is only readable inside the drop callback, so a port that hands + /// over a lazy view of it delivers empty data by the time the event dispatch thread reads + /// it. + /// + /// #### Parameters + /// + /// - `windowId`: the id of the window dropped on, or zero for the main surface + /// + /// - `x`: the pointer position within that surface + /// + /// - `y`: the pointer position within that surface + /// + /// - `content`: the dropped representations + /// + /// - `action`: the action the operating system settled on + /// + /// #### Returns + /// + /// the action actually accepted, or `NativeDragOperation#ACTION_NONE` when nothing under + /// the pointer took the drop and the port should report the transfer as failed + public static int drop(int windowId, int x, int y, ClipboardContent content, int action) { + Component target = findTarget(windowId, x, y, content); + currentTarget = null; + overDispatchPending = false; + if (target == null) { + currentAction = NativeDragOperation.ACTION_NONE; + return NativeDragOperation.ACTION_NONE; + } + int accepted = action & target.getAcceptedDropActions(); + if (accepted == NativeDragOperation.ACTION_NONE) { + currentAction = NativeDragOperation.ACTION_NONE; + return NativeDragOperation.ACTION_NONE; + } + accepted = preferredAction(accepted); + currentAction = accepted; + dispatch(target, ActionEvent.Type.NativeDrop, content, x, y, accepted); + return accepted; + } + + /// Reports that the session started by `#startDrag(com.codename1.ui.Component, + /// com.codename1.ui.NativeDragOperation)` has finished, whatever the outcome, so that a + /// source offering `NativeDragOperation#ACTION_MOVE` learns whether to delete its copy. + /// + /// #### Parameters + /// + /// - `performedAction`: the action the receiver performed, or + /// `NativeDragOperation#ACTION_NONE` when the drag was cancelled or refused + public static void dragCompleted(final int performedAction) { + final NativeDragOperation op = active; + active = null; + currentTarget = null; + currentAction = NativeDragOperation.ACTION_NONE; + overDispatchPending = false; + if (op == null) { + return; + } + Display.getInstance().callSerially(new Runnable() { + public void run() { + op.fireCompleted(performedAction); + } + }); + } + + // ------------------------------------------------------------------------------------ + + /// Resolves the deepest component under the pointer that is willing to take this content. + /// + /// Runs on the native drag thread and reads the component tree without mutating it, which + /// is the same thing the ports already do to route a native pointer press. + private static Component findTarget(int windowId, int x, int y, ClipboardContent content) { + Container root = surfaceFor(windowId); + if (root == null) { + return null; + } + Component cmp; + try { + cmp = root.getComponentAt(x, y); + } catch (Throwable err) { + // The tree can be mutated on the event dispatch thread while this walks it. A drag + // event that lands mid-layout is not worth a crash; the next one resolves. + return null; + } + while (cmp != null) { + if (cmp.isNativeDropTarget() && !cmp.isIgnorePointerEvents() && cmp.isEnabled()) { + try { + if (cmp.canAcceptNativeDrop(content)) { + return cmp; + } + } catch (Throwable err) { + Log.e(err); + } + } + cmp = cmp.getParent(); + } + return null; + } + + /// The container a window id names: the current form for the main surface, the window + /// itself otherwise. + private static Container surfaceFor(int windowId) { + if (windowId == 0) { + return Display.getInstance().getCurrent(); + } + try { + TopLevelContainer top = Desktop.getInstance().windowById(windowId); + return top == null ? null : top.asContainer(); + } catch (Throwable err) { + return null; + } + } + + /// Picks one action out of a bit set, preferring a copy because it is the one that cannot + /// destroy the source's data. + private static int preferredAction(int actions) { + if ((actions & NativeDragOperation.ACTION_COPY) != 0) { + return NativeDragOperation.ACTION_COPY; + } + if ((actions & NativeDragOperation.ACTION_MOVE) != 0) { + return NativeDragOperation.ACTION_MOVE; + } + if ((actions & NativeDragOperation.ACTION_LINK) != 0) { + return NativeDragOperation.ACTION_LINK; + } + return NativeDragOperation.ACTION_NONE; + } + + /// Queues one callback onto the event dispatch thread, where component code is allowed to + /// run, and folds whatever the target decided back into the answer the next drag event will + /// give the operating system. + private static void dispatch(final Component target, final ActionEvent.Type type, + final ClipboardContent content, final int x, final int y, final int allowedActions) { + if (target == null) { + if (type == ActionEvent.Type.NativeDragOver) { + overDispatchPending = false; + } + return; + } + final boolean local = active != null; + Display.getInstance().callSerially(new Runnable() { + public void run() { + try { + NativeDropEvent ev = new NativeDropEvent(target, type, content, x, y, allowedActions, local); + if (type == ActionEvent.Type.NativeDragOver || type == ActionEvent.Type.NativeDragEnter) { + // The target starts from what the framework already agreed to, so a + // target that does not care keeps the answer stable instead of + // resetting it to the default on every event. + ev.accept(currentAction); + } + target.dispatchNativeDropEvent(ev); + if (type == ActionEvent.Type.NativeDragOver || type == ActionEvent.Type.NativeDragEnter) { + if (currentTarget == target) { // NOPMD CompareObjectsWithEquals + currentAction = ev.getAcceptedAction(); + } + } + } catch (Throwable err) { + Log.e(err); + } finally { + if (type == ActionEvent.Type.NativeDragOver) { + overDispatchPending = false; + } + } + } + }); + } +} diff --git a/CodenameOne/src/com/codename1/ui/NativeDragOperation.java b/CodenameOne/src/com/codename1/ui/NativeDragOperation.java new file mode 100644 index 00000000000..49f43ddfd66 --- /dev/null +++ b/CodenameOne/src/com/codename1/ui/NativeDragOperation.java @@ -0,0 +1,265 @@ +/* + * Copyright (c) 2026, Codename One and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. Codename One designates this + * particular file as subject to the "Classpath" exception as provided + * by Oracle in the LICENSE file that accompanied this code. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Codename One through http://www.codenameone.com/ if you + * need additional information or have any questions. + */ + + +package com.codename1.ui; + +import com.codename1.ui.events.ActionEvent; +import com.codename1.ui.events.ActionListener; +import com.codename1.ui.util.EventDispatcher; + +/// Everything the operating system needs in order to drag something out of a Codename One +/// component: what is being dragged, what the receiver is allowed to do with it, and what the +/// user should see under the cursor while dragging. +/// +/// The payload is a `ClipboardContent`, the same object a copy publishes, so a component that +/// can already be copied can be made draggable by handing the very same content to +/// `Component#setNativeDragOperation(com.codename1.ui.NativeDragOperation)`. Offering several +/// representations is what lets one drag land correctly in unrelated applications: a text editor +/// takes `ClipboardContent#MIME_HTML`, a plain text field takes `ClipboardContent#MIME_TEXT` and +/// the desktop or a file manager takes `ClipboardContent#MIME_FILE`. +/// +/// Representations that are expensive to produce -- the file that only exists if the user +/// actually drops on the desktop -- should be registered with +/// `ClipboardContent#setDataProvider(java.lang.String, com.codename1.ui.ClipboardDataProvider)` +/// rather than built when the drag starts. +/// +/// #### Moving rather than copying +/// +/// `#ACTION_MOVE` means the receiver takes ownership and the source is expected to delete its +/// copy. The source only learns whether that happened once the operating system has finished +/// the transfer, which is why the outcome arrives asynchronously through +/// `#addCompletionListener(com.codename1.ui.events.ActionListener)` and not from the call that +/// started the drag. +public class NativeDragOperation { + /// No transfer, which is what a rejected or cancelled drag reports. + public static final int ACTION_NONE = 0; + + /// The receiver takes a copy and the source keeps its own. + public static final int ACTION_COPY = 1; + + /// The receiver takes ownership; the source should delete its copy when the drag completes + /// with this action. + public static final int ACTION_MOVE = 2; + + /// The receiver stores a reference rather than the data, the way a shortcut or an alias does. + public static final int ACTION_LINK = 4; + + private final ClipboardContent content; + private int allowedActions = ACTION_COPY; + private Image dragImage; + private int dragImageOffsetX; + private int dragImageOffsetY; + private String label; + private int performedAction = ACTION_NONE; + private Component source; + private EventDispatcher completionListeners; + + /// Creates a drag carrying the given representations. + /// + /// #### Parameters + /// + /// - `content`: the payload, which must not be null + public NativeDragOperation(ClipboardContent content) { + if (content == null) { + throw new IllegalArgumentException("content must not be null"); + } + this.content = content; + } + + /// Creates a plain text drag, the shorthand for the common case. + /// + /// #### Parameters + /// + /// - `text`: the text being dragged + public NativeDragOperation(String text) { + this(new ClipboardContent().setData(ClipboardContent.MIME_TEXT, text == null ? "" : text)); + } + + /// Creates a drag carrying files, which is what a drop onto the desktop or a file manager + /// consumes. + /// + /// #### Parameters + /// + /// - `paths`: the file paths or `file:` URIs being dragged + /// + /// #### Returns + /// + /// the new operation + public static NativeDragOperation createFileDrag(String[] paths) { + return new NativeDragOperation(new ClipboardContent().setFiles(paths)); + } + + /// Returns the payload. + public ClipboardContent getContent() { + return content; + } + + /// Returns the bit set of actions the source is willing to allow, `#ACTION_COPY` by default. + public int getAllowedActions() { + return allowedActions; + } + + /// Sets the bit set of actions the source is willing to allow. The receiver chooses one of + /// them, usually influenced by the modifier keys the user is holding. + /// + /// #### Parameters + /// + /// - `allowedActions`: any combination of `#ACTION_COPY`, `#ACTION_MOVE` and `#ACTION_LINK` + /// + /// #### Returns + /// + /// this instance, for chaining + public NativeDragOperation setAllowedActions(int allowedActions) { + this.allowedActions = allowedActions; + return this; + } + + /// Returns the image drawn under the cursor during the drag, or null to let the port draw + /// the component itself. + public Image getDragImage() { + return dragImage; + } + + /// Sets the image drawn under the cursor during the drag. When this is left null the port + /// renders the dragged component through `Component#getDragImage()`, so the user sees the + /// thing they grabbed. + /// + /// #### Parameters + /// + /// - `dragImage`: the image, or null for the default + /// + /// #### Returns + /// + /// this instance, for chaining + public NativeDragOperation setDragImage(Image dragImage) { + this.dragImage = dragImage; + return this; + } + + /// Returns the x offset of the cursor within the drag image. + public int getDragImageOffsetX() { + return dragImageOffsetX; + } + + /// Returns the y offset of the cursor within the drag image. + public int getDragImageOffsetY() { + return dragImageOffsetY; + } + + /// Places the cursor at a specific point of the drag image, so the image keeps the position + /// it had relative to the finger or pointer when the drag began. + /// + /// #### Parameters + /// + /// - `x`: the x offset within the image + /// + /// - `y`: the y offset within the image + /// + /// #### Returns + /// + /// this instance, for chaining + public NativeDragOperation setDragImageOffset(int x, int y) { + this.dragImageOffsetX = x; + this.dragImageOffsetY = y; + return this; + } + + /// Returns the human readable label some platforms show beside the drag image. + public String getLabel() { + return label; + } + + /// Sets the human readable label some platforms show beside the drag image, such as the + /// file name of a dragged document. Platforms that have no such affordance ignore it. + /// + /// #### Parameters + /// + /// - `label`: the label + /// + /// #### Returns + /// + /// this instance, for chaining + public NativeDragOperation setLabel(String label) { + this.label = label; + return this; + } + + /// Returns the component the drag started from, or null when the drag was started through + /// `NativeDragAndDrop#startDrag(com.codename1.ui.Component, com.codename1.ui.NativeDragOperation)` + /// without one. + public Component getSource() { + return source; + } + + void setSource(Component source) { + this.source = source; + } + + /// Returns the action the receiver actually performed, valid once the drag has completed. + /// Before that, and for a drag that was cancelled or rejected, this is `#ACTION_NONE`. + public int getPerformedAction() { + return performedAction; + } + + /// Adds a listener notified on the event dispatch thread once the operating system has + /// finished with this drag, whether it was dropped or abandoned. Read + /// `#getPerformedAction()` from the listener; a source offering `#ACTION_MOVE` deletes its + /// copy here and nowhere else, because until this fires nothing is known about whether the + /// receiver took it. + /// + /// #### Parameters + /// + /// - `l`: the listener + public void addCompletionListener(ActionListener l) { + if (completionListeners == null) { + completionListeners = new EventDispatcher(); + } + completionListeners.addListener(l); + } + + /// Removes a listener added by + /// `#addCompletionListener(com.codename1.ui.events.ActionListener)`. + /// + /// #### Parameters + /// + /// - `l`: the listener + public void removeCompletionListener(ActionListener l) { + if (completionListeners != null) { + completionListeners.removeListener(l); + } + } + + /// Records the outcome and notifies the completion listeners. Invoked by the port, on the + /// event dispatch thread, when the native drag session ends. + /// + /// #### Parameters + /// + /// - `action`: the action the receiver performed, or `#ACTION_NONE` + void fireCompleted(int action) { + performedAction = action; + if (completionListeners != null && completionListeners.hasListeners()) { + completionListeners.fireActionEvent(new ActionEvent(this, ActionEvent.Type.NativeDragCompleted)); + } + } +} diff --git a/CodenameOne/src/com/codename1/ui/NativeDropEvent.java b/CodenameOne/src/com/codename1/ui/NativeDropEvent.java new file mode 100644 index 00000000000..2ab27074a1f --- /dev/null +++ b/CodenameOne/src/com/codename1/ui/NativeDropEvent.java @@ -0,0 +1,154 @@ +/* + * Copyright (c) 2026, Codename One and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. Codename One designates this + * particular file as subject to the "Classpath" exception as provided + * by Oracle in the LICENSE file that accompanied this code. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Codename One through http://www.codenameone.com/ if you + * need additional information or have any questions. + */ + + +package com.codename1.ui; + +import com.codename1.ui.events.ActionEvent; + +/// One step of a native operating system drag passing over, or landing on, a component that was +/// marked as a native drop target with `Component#setNativeDropTarget(boolean)`. +/// +/// The payload is a `ClipboardContent` -- the same shape a paste produces -- so a component that +/// already knows how to paste knows how to accept a drop. Inspect +/// `ClipboardContent#getMimeTypes()` and take the richest representation you understand, exactly +/// as you would for a paste. +/// +/// The x and y inherited from `ActionEvent` are absolute screen coordinates within the surface +/// the drag is over, so `Component#getAbsoluteX()` and `Component#getAbsoluteY()` convert them +/// to component coordinates. +/// +/// #### Accepting +/// +/// A drag event tells the operating system whether the drop would be allowed and what would +/// happen. Call `#accept(int)` with one of the actions in `#getAllowedActions()` to show the +/// user the corresponding cursor, or `#reject()` to refuse. A target that never calls either +/// accepts `NativeDragOperation#ACTION_COPY` when the source allows it, which is what most +/// targets want. +public final class NativeDropEvent extends ActionEvent { + private final ClipboardContent content; + private final int allowedActions; + private final boolean local; + private final Component target; + private int acceptedAction; + + /// Creates a drop event. + /// + /// #### Parameters + /// + /// - `target`: the component the drag is over + /// + /// - `type`: the event type + /// + /// - `content`: the dragged payload + /// + /// - `x`: the absolute x position of the pointer + /// + /// - `y`: the absolute y position of the pointer + /// + /// - `allowedActions`: the actions the drag source permits + /// + /// - `local`: true when the drag started inside this application + NativeDropEvent(Component target, Type type, ClipboardContent content, int x, int y, + int allowedActions, boolean local) { + super(target, type, x, y); + this.target = target; + this.content = content; + this.allowedActions = allowedActions; + this.local = local; + this.acceptedAction = defaultAction(allowedActions); + } + + /// Picks the action a target that expresses no preference gets: a copy when the source + /// allows one, otherwise whichever single action it does allow. + private static int defaultAction(int allowedActions) { + if ((allowedActions & NativeDragOperation.ACTION_COPY) != 0) { + return NativeDragOperation.ACTION_COPY; + } + if ((allowedActions & NativeDragOperation.ACTION_MOVE) != 0) { + return NativeDragOperation.ACTION_MOVE; + } + if ((allowedActions & NativeDragOperation.ACTION_LINK) != 0) { + return NativeDragOperation.ACTION_LINK; + } + return NativeDragOperation.ACTION_NONE; + } + + /// Returns the dragged payload. + public ClipboardContent getContent() { + return content; + } + + /// Returns the component the drag is over. + public Component getTarget() { + return target; + } + + /// Returns the bit set of actions the drag source permits. + public int getAllowedActions() { + return allowedActions; + } + + /// Returns true when the drag started inside this application rather than in another + /// application or on the desktop. A target that reorders its own items usually only wants + /// to handle local drags, and a target that imports foreign data usually only wants the + /// rest. + public boolean isLocal() { + return local; + } + + /// Returns the file paths carried by the drag, or null when it carries none. This is the + /// representation a drag out of a file manager or off the desktop arrives with. + public String[] getFiles() { + return content == null ? null : content.getFiles(); + } + + /// Returns the plain text carried by the drag, or null when it carries none. + public String getText() { + return content == null ? null : content.getText(ClipboardContent.MIME_TEXT); + } + + /// Returns the action this target has accepted, or `NativeDragOperation#ACTION_NONE` when + /// it has refused the drop. + public int getAcceptedAction() { + return acceptedAction; + } + + /// Accepts the drag, telling the operating system what dropping here would do. An action + /// the source does not allow is refused rather than silently substituted, because showing + /// the user a move cursor for a drag that can only copy is worse than showing no cursor. + /// + /// #### Parameters + /// + /// - `action`: one of `NativeDragOperation#ACTION_COPY`, `NativeDragOperation#ACTION_MOVE` + /// or `NativeDragOperation#ACTION_LINK` + public void accept(int action) { + acceptedAction = (action & allowedActions) == action ? action : NativeDragOperation.ACTION_NONE; + } + + /// Refuses the drag, so the user sees a "no drop" cursor over this component and no drop + /// event is delivered. + public void reject() { + acceptedAction = NativeDragOperation.ACTION_NONE; + } +} diff --git a/CodenameOne/src/com/codename1/ui/Window.java b/CodenameOne/src/com/codename1/ui/Window.java index c7e104db000..8b2e9a2c315 100644 --- a/CodenameOne/src/com/codename1/ui/Window.java +++ b/CodenameOne/src/com/codename1/ui/Window.java @@ -3320,6 +3320,12 @@ public void pointerDragged(int x, int y) { stylusCmp.fireStylusEvent(ActionEvent.Type.PointerDrag, x, y); } } + // A press that landed on a native drag source becomes an operating system drag + // here, exactly as it does on a Form, once it has moved far enough to be a drag + // rather than a click. The platform owns the gesture from that point. + if (NativeDragAndDrop.pointerDragged(x, y)) { + return; + } // Read and cleared here, exactly as Form does: the flag describes the drag that // took a momentum scroll over, and leaving it set would tell every later drag in // the session that it too continued out of a glide. @@ -3386,6 +3392,9 @@ public void pointerReleased(int x, int y) { stylusCmp.fireStylusEvent(ActionEvent.Type.PointerReleased, x, y); } } + // A press that never became a drag releases the operation the press staged, so a + // later gesture somewhere else cannot start the drag this one declined to. + NativeDragAndDrop.pointerReleased(); // The token identifying *this* gesture. A release handler may enter // invokeAndBlock, whose nested event loop can dispatch a fresh press in this // same window before the handler returns; tearing down unconditionally then diff --git a/CodenameOne/src/com/codename1/ui/events/ActionEvent.java b/CodenameOne/src/com/codename1/ui/events/ActionEvent.java index 2bbe7cc3fce..1c1d4343394 100644 --- a/CodenameOne/src/com/codename1/ui/events/ActionEvent.java +++ b/CodenameOne/src/com/codename1/ui/events/ActionEvent.java @@ -569,5 +569,22 @@ public enum Type { OpenGallery, IsGalleryTypeSupported, + + /// Fired when a native operating system drag enters a component that is a native drop + /// target, see `com.codename1.ui.Component#addNativeDropListener(com.codename1.ui.events.ActionListener)` + NativeDragEnter, + + /// Fired repeatedly while a native operating system drag moves over a native drop target + NativeDragOver, + + /// Fired when a native operating system drag leaves a native drop target + NativeDragExit, + + /// Fired when a native operating system drag is dropped on a native drop target + NativeDrop, + + /// Fired on the source of a native drag once the operating system has finished the + /// transfer, telling the source which action -- if any -- was actually performed + NativeDragCompleted, } } diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java index 21660be5870..925f7902cb8 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java @@ -2192,6 +2192,9 @@ private void initSurface() { myView = new AndroidAsyncView(getActivity(), AndroidImplementation.this); } myView.getAndroidView().setVisibility(View.VISIBLE); + // Makes the surface an Android drop target, so a drag from another application -- + // or from elsewhere in this one -- reaches the components that asked for it. + AndroidNativeDragAndDrop.install(this, myView.getAndroidView()); if (hideOverlayWindowsRequested) { setHideOverlayWindows(true); @@ -10156,29 +10159,11 @@ public void run() { clipboard.setText(obj.toString()); } else { android.content.ClipboardManager clipboard = (android.content.ClipboardManager) getActivity().getSystemService(Context.CLIPBOARD_SERVICE); - android.content.ClipData clip = null; - if (sdk >= 16 && obj instanceof ClipboardContent - && ((ClipboardContent) obj).getText(ClipboardContent.MIME_HTML) != null) { - ClipboardContent rich = (ClipboardContent) obj; - clip = ClipData.newHtmlText("Codename One", - rich.getText(ClipboardContent.MIME_TEXT), - rich.getText(ClipboardContent.MIME_HTML)); - } else if (obj instanceof ClipboardContent - && ((ClipboardContent) obj).getText(ClipboardContent.MIME_TEXT) != null) { - clip = ClipData.newPlainText("Codename One", - ((ClipboardContent)obj).getText(ClipboardContent.MIME_TEXT)); - } else if (!(obj instanceof ClipboardContent)) { - clip = ClipData.newPlainText("Codename One", obj.toString()); - } + android.content.ClipData clip; if (obj instanceof ClipboardContent) { - try { - clip = enrichClipWithBinaryContent((ClipboardContent) obj, clip); - } catch (Throwable t) { - com.codename1.io.Log.e(t); - } - } - if (clip == null) { - clip = ClipData.newPlainText("Codename One", ""); + clip = clipDataFor((ClipboardContent) obj); + } else { + clip = ClipData.newPlainText("Codename One", obj.toString()); } clipboard.setPrimaryClip(clip); } @@ -10186,6 +10171,65 @@ public void run() { }); } + /// Builds the Android clip that publishes a `ClipboardContent`, for a clipboard copy and + /// for a native drag alike -- both hand another application the same thing, so both go + /// through the same conversion, including the file provider URIs that let the receiving + /// application read generated image bytes. + /// + /// #### Parameters + /// + /// - `content`: the representations to publish + /// + /// #### Returns + /// + /// the clip, never null + ClipData clipDataFor(ClipboardContent content) { + int sdk = android.os.Build.VERSION.SDK_INT; + ClipData clip = null; + if (sdk >= 16 && content.getText(ClipboardContent.MIME_HTML) != null) { + clip = ClipData.newHtmlText("Codename One", + content.getText(ClipboardContent.MIME_TEXT), + content.getText(ClipboardContent.MIME_HTML)); + } else if (content.getText(ClipboardContent.MIME_TEXT) != null) { + clip = ClipData.newPlainText("Codename One", content.getText(ClipboardContent.MIME_TEXT)); + } + try { + clip = enrichClipWithBinaryContent(content, clip); + } catch (Throwable t) { + com.codename1.io.Log.e(t); + } + if (clip == null) { + clip = ClipData.newPlainText("Codename One", ""); + } + return clip; + } + + // ------------------------------------------------------------------------------------ + // Native drag and drop. See AndroidNativeDragAndDrop; the payload is the same ClipData a + // copy publishes, which is why a drag out of the application lands in another application + // exactly as a paste would. + // ------------------------------------------------------------------------------------ + + @Override + public boolean isNativeDragAndDropSupported() { + return AndroidNativeDragAndDrop.isSupported(); + } + + @Override + public boolean isNativeDragOutsideApplicationSupported() { + return AndroidNativeDragAndDrop.isOutsideApplicationSupported(); + } + + @Override + public boolean startNativeDrag(com.codename1.ui.NativeDragOperation op) { + return AndroidNativeDragAndDrop.startDrag(this, op); + } + + @Override + public void cancelNativeDrag() { + AndroidNativeDragAndDrop.cancelDrag(); + } + /** * Enriches the given base ClipData (which may be null) with image bytes and/or file * references carried by the ClipboardContent, exposing binary content as FileProvider @@ -10307,62 +10351,12 @@ public void run() { if (clip == null || clip.getItemCount() == 0) { return; } - ClipboardContent content = new ClipboardContent(); - String plain = null; - String html = null; - boolean hasImage = false; - List fileUris = new ArrayList(); - for (int i = 0; i < clip.getItemCount(); i++) { - ClipData.Item item = clip.getItemAt(i); - try { - Uri uri = item.getUri(); - if (uri != null) { - String type = getContext().getContentResolver().getType(uri); - if (type != null && type.startsWith("image/")) { - InputStream in = getContext().getContentResolver().openInputStream(uri); - if (in != null) { - try { - byte[] bytes = Util.readInputStream(in); - content.setData(mimeForImageType(type), bytes); - hasImage = true; - } finally { - in.close(); - } - } - continue; - } - // Non-image URI -> file reference - fileUris.add(uri.toString()); - continue; - } - } catch (Throwable t) { - com.codename1.io.Log.e(t); - } - if (html == null && sdk >= 16) { - String itemHtml = item.getHtmlText(); - if (itemHtml != null && itemHtml.length() > 0) { - html = itemHtml; - } - } - if (plain == null) { - CharSequence text = item.coerceToText(getContext()); - if (text != null && text.length() > 0) { - plain = text.toString(); - } - } - } - if (html != null) { - content.setData(ClipboardContent.MIME_HTML, html); - } - if (!fileUris.isEmpty()) { - content.setData(ClipboardContent.MIME_FILE, - fileUris.size() == 1 ? (Object) fileUris.get(0) : (Object) fileUris.toArray(new String[0])); - } - content.setData(ClipboardContent.MIME_TEXT, plain == null ? "" : plain); - if (hasImage || html != null || !fileUris.isEmpty()) { + ClipboardContent content = contentFromClip(clip); + String plain = content.getText(ClipboardContent.MIME_TEXT); + if (content.getMimeTypes().length > 1) { response[0] = content; } else { - response[0] = plain; + response[0] = plain != null && plain.length() > 0 ? plain : null; } } } @@ -10370,6 +10364,79 @@ public void run() { return response[0]; } + /// Reads an Android `android.content.ClipData` into the framework's `ClipboardContent`. + /// + /// Shared by paste and by a native drop, because Android describes both the same way: a + /// list of items that are each text, HTML or a URI, and a URI is either an image to be read + /// or a file reference to be passed along. The plain text representation is always present, + /// even when empty, so a caller can tell "nothing but text" from "something richer" by the + /// number of MIME types. + /// + /// #### Parameters + /// + /// - `clip`: the clip data, which may be null + /// + /// #### Returns + /// + /// the content, never null + ClipboardContent contentFromClip(ClipData clip) { + ClipboardContent content = new ClipboardContent(); + if (clip == null) { + content.setData(ClipboardContent.MIME_TEXT, ""); + return content; + } + int sdk = android.os.Build.VERSION.SDK_INT; + String plain = null; + String html = null; + List fileUris = new ArrayList(); + for (int i = 0; i < clip.getItemCount(); i++) { + ClipData.Item item = clip.getItemAt(i); + try { + Uri uri = item.getUri(); + if (uri != null) { + String type = getContext().getContentResolver().getType(uri); + if (type != null && type.startsWith("image/")) { + InputStream in = getContext().getContentResolver().openInputStream(uri); + if (in != null) { + try { + byte[] bytes = Util.readInputStream(in); + content.setData(mimeForImageType(type), bytes); + } finally { + in.close(); + } + } + continue; + } + // Non-image URI -> file reference + fileUris.add(uri.toString()); + continue; + } + } catch (Throwable t) { + com.codename1.io.Log.e(t); + } + if (html == null && sdk >= 16) { + String itemHtml = item.getHtmlText(); + if (itemHtml != null && itemHtml.length() > 0) { + html = itemHtml; + } + } + if (plain == null) { + CharSequence text = item.coerceToText(getContext()); + if (text != null && text.length() > 0) { + plain = text.toString(); + } + } + } + if (html != null) { + content.setData(ClipboardContent.MIME_HTML, html); + } + if (!fileUris.isEmpty()) { + content.setFiles(fileUris.toArray(new String[fileUris.size()])); + } + content.setData(ClipboardContent.MIME_TEXT, plain == null ? "" : plain); + return content; + } + public static MediaException createMediaException(int extra) { MediaErrorType type; String message; diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java b/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java new file mode 100644 index 00000000000..8fbc79b30bb --- /dev/null +++ b/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java @@ -0,0 +1,308 @@ +/* + * Copyright (c) 2026, Codename One and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. Codename One designates this + * particular file as subject to the "Classpath" exception as provided + * by Oracle in the LICENSE file that accompanied this code. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Codename One through http://www.codenameone.com/ if you + * need additional information or have any questions. + */ + + +package com.codename1.impl.android; + +import android.content.ClipData; +import android.content.ClipDescription; +import android.graphics.Bitmap; +import android.graphics.Canvas; +import android.graphics.Point; +import android.os.Build; +import android.view.DragEvent; +import android.view.View; + +import com.codename1.io.Log; +import com.codename1.ui.ClipboardContent; +import com.codename1.ui.ClipboardDataProvider; +import com.codename1.ui.NativeDragAndDrop; +import com.codename1.ui.NativeDragOperation; + +/// Android's drag and drop, wired to the framework's. +/// +/// Android has had in-application drag and drop since Ice Cream Sandwich, but a drag could not +/// leave the application until Nougat added `View#DRAG_FLAG_GLOBAL`. That is the split reported +/// by `#isSupported()` and `#isOutsideApplicationSupported()`: on a phone before Nougat a drag +/// can still move things around inside the application, and on a tablet or a Chromebook running +/// Nougat or later it can be dropped into another application beside it. +/// +/// #### Files +/// +/// A dragged file travels as a `content:` URI rather than a path, which is also how the +/// clipboard carries one, so the same `AndroidImplementation#contentFromClip(android.content.ClipData)` +/// reader serves both. A URI from another application is only readable while the drop's +/// permission grant is held, which is why the content is read inside the drop callback rather +/// than handed to the event dispatch thread to read later. +class AndroidNativeDragAndDrop { + /// The operation currently being dragged out of this application, so that the outcome + /// reported when the drag ends can be attributed to it. + private static NativeDragOperation exporting; + + /// The action last agreed with the framework, reported back when the drop is accepted. + /// Android's drag events carry no copy/move/link distinction of their own. + private static int lastAction = NativeDragOperation.ACTION_NONE; + + private AndroidNativeDragAndDrop() { + } + + /// Returns true when this device can drag and drop at all. + static boolean isSupported() { + return Build.VERSION.SDK_INT >= 11; + } + + /// Returns true when a drag started here can be dropped into another application, which + /// Android only allows from Nougat onwards. + static boolean isOutsideApplicationSupported() { + return Build.VERSION.SDK_INT >= 24; + } + + /// Makes the Codename One surface a drop target. Called once, as the view is created. + /// + /// #### Parameters + /// + /// - `impl`: the implementation, used to read dropped content + /// + /// - `view`: the Android view Codename One renders into + static void install(final AndroidImplementation impl, final View view) { + if (!isSupported() || view == null) { + return; + } + try { + view.setOnDragListener(new View.OnDragListener() { + public boolean onDrag(View v, DragEvent event) { + return handle(impl, v, event); + } + }); + } catch (Throwable err) { + Log.e(err); + } + } + + /// Starts an Android drag for the operation the framework decided on. Invoked on the + /// Codename One event dispatch thread; the drag itself has to begin on the Android UI + /// thread, which is what the post below is for. + static boolean startDrag(final AndroidImplementation impl, final NativeDragOperation op) { + if (op == null || !isSupported()) { + return false; + } + final CodenameOneSurface surface = impl.myView; + final View view = surface == null ? null : surface.getAndroidView(); + if (view == null) { + return false; + } + final ClipData clip = toClipData(impl, op.getContent()); + if (clip == null) { + return false; + } + exporting = op; + lastAction = NativeDragOperation.ACTION_NONE; + view.post(new Runnable() { + public void run() { + boolean started = false; + try { + View.DragShadowBuilder shadow = shadowFor(view, op); + if (Build.VERSION.SDK_INT >= 24) { + int flags = View.DRAG_FLAG_GLOBAL | View.DRAG_FLAG_GLOBAL_URI_READ; + started = view.startDragAndDrop(clip, shadow, null, flags); + } else { + started = view.startDrag(clip, shadow, null, 0); + } + } catch (Throwable err) { + Log.e(err); + } + if (!started) { + exporting = null; + NativeDragAndDrop.dragCompleted(NativeDragOperation.ACTION_NONE); + } + } + }); + return true; + } + + /// Forgets a prepared operation because the press turned out to be a click. + static void cancelDrag() { + exporting = null; + } + + // ------------------------------------------------------------------------------------ + + private static boolean handle(AndroidImplementation impl, View view, DragEvent event) { + try { + switch (event.getAction()) { + case DragEvent.ACTION_DRAG_STARTED: + // Returning true is what subscribes this view to the rest of the drag; + // a view that answers false here never sees another event, so the answer + // is unconditional and the real filtering happens per position below. + return true; + case DragEvent.ACTION_DRAG_ENTERED: + lastAction = NativeDragAndDrop.dragEnter(0, (int) event.getX(), (int) event.getY(), + describe(event.getClipDescription()), allowedActions()); + return true; + case DragEvent.ACTION_DRAG_LOCATION: + lastAction = NativeDragAndDrop.dragOver(0, (int) event.getX(), (int) event.getY(), + describe(event.getClipDescription()), allowedActions()); + return true; + case DragEvent.ACTION_DRAG_EXITED: + NativeDragAndDrop.dragExit(0); + lastAction = NativeDragOperation.ACTION_NONE; + return true; + case DragEvent.ACTION_DROP: + return drop(impl, view, event); + case DragEvent.ACTION_DRAG_ENDED: + if (exporting != null) { + exporting = null; + NativeDragAndDrop.dragCompleted(event.getResult() + ? preferred(allowedActions()) : NativeDragOperation.ACTION_NONE); + } + lastAction = NativeDragOperation.ACTION_NONE; + return true; + default: + return false; + } + } catch (Throwable err) { + Log.e(err); + return false; + } + } + + private static boolean drop(AndroidImplementation impl, View view, DragEvent event) { + // A URI dropped by another application is only readable while this grant is held, and + // the grant only exists from here on. Reading the content inside this method rather + // than on the event dispatch thread is what keeps a dropped file readable. + if (Build.VERSION.SDK_INT >= 24 && impl.getActivity() != null) { + try { + impl.getActivity().requestDragAndDropPermissions(event); + } catch (Throwable err) { + // A drag from within this application needs no grant and refuses one. + Log.e(err); + } + } + ClipboardContent content = impl.contentFromClip(event.getClipData()); + int action = lastAction == NativeDragOperation.ACTION_NONE + ? preferred(allowedActions()) : lastAction; + int accepted = NativeDragAndDrop.drop(0, (int) event.getX(), (int) event.getY(), content, action); + lastAction = NativeDragOperation.ACTION_NONE; + return accepted != NativeDragOperation.ACTION_NONE; + } + + /// The actions in play. A drag this application started offers whatever its source allowed; + /// one arriving from another application is a copy, because Android's cross-application + /// drag has no way to express anything else. + private static int allowedActions() { + NativeDragOperation op = exporting; + return op == null ? NativeDragOperation.ACTION_COPY : op.getAllowedActions(); + } + + private static int preferred(int actions) { + if ((actions & NativeDragOperation.ACTION_COPY) != 0) { + return NativeDragOperation.ACTION_COPY; + } + if ((actions & NativeDragOperation.ACTION_MOVE) != 0) { + return NativeDragOperation.ACTION_MOVE; + } + if ((actions & NativeDragOperation.ACTION_LINK) != 0) { + return NativeDragOperation.ACTION_LINK; + } + return NativeDragOperation.ACTION_NONE; + } + + /// Describes a drag in progress from its MIME types alone. + /// + /// Android does not hand over the data until the drop, so every representation here is a + /// provider that answers null. That is enough: a drop target decides whether it wants the + /// drag from the MIME types, and the real content arrives on the drop. + private static ClipboardContent describe(ClipDescription description) { + ClipboardContent content = new ClipboardContent(); + if (description == null) { + return content; + } + for (int iter = 0; iter < description.getMimeTypeCount(); iter++) { + String mime = description.getMimeType(iter); + if (mime == null) { + continue; + } + mime = mime.toLowerCase(); + if ("text/uri-list".equals(mime)) { + // Android carries a dragged file as a URI, which is what the framework calls a + // file list; advertise both so either kind of target matches. + declare(content, ClipboardContent.MIME_FILE); + declare(content, ClipboardContent.MIME_URI_LIST); + continue; + } + declare(content, mime); + } + return content; + } + + private static void declare(ClipboardContent content, String mime) { + if (content.hasMimeType(mime)) { + return; + } + content.setDataProvider(mime, new ClipboardDataProvider() { + public Object getClipboardData(String requested) { + // Android reveals nothing until the drop; the drop callback replaces this with + // the real content. + return null; + } + }); + } + + /// Builds the Android clip for an outgoing drag. The clipboard already knows how to turn a + /// `ClipboardContent` into a clip -- including writing image bytes out through the + /// application's file provider so another application can read them -- so this reuses that + /// rather than growing a second conversion that would drift from it. + private static ClipData toClipData(AndroidImplementation impl, ClipboardContent content) { + try { + return impl.clipDataFor(content); + } catch (Throwable err) { + Log.e(err); + return null; + } + } + + /// The image under the finger during the drag: whatever the operation supplied, rendered at + /// the offset the gesture grabbed it by. + private static View.DragShadowBuilder shadowFor(View view, NativeDragOperation op) { + com.codename1.ui.Image image = op.getDragImage(); + Object peer = image == null ? null : image.getImage(); + if (!(peer instanceof Bitmap)) { + return new View.DragShadowBuilder(view); + } + final Bitmap bitmap = (Bitmap) peer; + final int touchX = Math.max(0, Math.min(bitmap.getWidth(), op.getDragImageOffsetX())); + final int touchY = Math.max(0, Math.min(bitmap.getHeight(), op.getDragImageOffsetY())); + return new View.DragShadowBuilder(view) { + @Override + public void onProvideShadowMetrics(Point size, Point touch) { + size.set(Math.max(1, bitmap.getWidth()), Math.max(1, bitmap.getHeight())); + touch.set(touchX, touchY); + } + + @Override + public void onDrawShadow(Canvas canvas) { + canvas.drawBitmap(bitmap, 0, 0, null); + } + }; + } +} diff --git a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java new file mode 100644 index 00000000000..5fc6a77f68e --- /dev/null +++ b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java @@ -0,0 +1,550 @@ +/* + * Copyright (c) 2026, Codename One and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. Codename One designates this + * particular file as subject to the "Classpath" exception as provided + * by Oracle in the LICENSE file that accompanied this code. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Codename One through http://www.codenameone.com/ if you + * need additional information or have any questions. + */ + + +package com.codename1.impl.javase; + +import com.codename1.io.Log; +import com.codename1.ui.ClipboardContent; +import com.codename1.ui.ClipboardDataProvider; +import com.codename1.ui.NativeDragAndDrop; +import com.codename1.ui.NativeDragOperation; + +import java.awt.EventQueue; +import java.awt.GraphicsEnvironment; +import java.awt.Point; +import java.awt.datatransfer.DataFlavor; +import java.awt.datatransfer.Transferable; +import java.awt.dnd.DnDConstants; +import java.awt.dnd.DropTarget; +import java.awt.dnd.DropTargetDragEvent; +import java.awt.dnd.DropTargetDropEvent; +import java.awt.dnd.DropTargetEvent; +import java.awt.dnd.DropTargetListener; +import java.awt.event.InputEvent; +import java.awt.event.MouseEvent; +import java.awt.image.BufferedImage; +import java.io.ByteArrayOutputStream; +import java.io.File; +import java.io.InputStream; +import java.net.URI; +import java.util.ArrayList; +import java.util.List; + +import javax.swing.JComponent; +import javax.swing.TransferHandler; + +/// Bridges Codename One's native drag and drop onto AWT's, which is what lets a desktop +/// application drag content out of itself -- onto the desktop, into a file manager, into another +/// application's window -- and accept drags coming the other way. +/// +/// #### Which thread does what +/// +/// AWT delivers drag notifications on its own event dispatch thread, which is not Codename +/// One's. Nothing here ever blocks one on the other: the Codename One event thread calls into +/// `javax.swing.TransferHandler` through `java.awt.EventQueue#invokeLater`, and the drop +/// callbacks answer AWT from `NativeDragAndDrop`, which resolves the target without needing the +/// Codename One event thread. That is not fastidiousness -- the Codename One event thread +/// blocks on AWT to blit every frame, so a synchronous call the other way deadlocks the +/// simulator on the first drag. +/// +/// #### Why the drag reads nothing until the drop +/// +/// While a drag is merely passing over the window the transferable's *data* is not reliably +/// readable -- on some platforms it does not exist yet -- but its list of flavors always is. So +/// a drag in progress is described by a `ClipboardContent` whose representations are all +/// `ClipboardDataProvider`s: enough for a drop target to say whether it wants a `text/html` or +/// a file list, without a byte being transferred for a drag that ends up going somewhere else. +/// On the drop the content is materialized eagerly instead, because the transferable stops +/// being readable the moment the drop callback returns. +final class JavaSENativeDragAndDrop { + /// The operation the Codename One event thread has asked to export, read by the transfer + /// handler on the AWT thread when the drag actually starts. One process drags one thing at + /// a time, so a single slot is the whole of the state. + private static volatile NativeDragOperation exporting; + + private JavaSENativeDragAndDrop() { + } + + /// Makes one canvas both an AWT drop target and a drag source. + /// + /// #### Parameters + /// + /// - `canvas`: the canvas, which is the surface for the main form or for one window + static void install(JavaSEPort.C canvas) { + if (GraphicsEnvironment.isHeadless()) { + return; + } + try { + canvas.setTransferHandler(new Cn1TransferHandler()); + new DropTarget(canvas, DnDConstants.ACTION_COPY_OR_MOVE | DnDConstants.ACTION_LINK, + new Cn1DropTargetListener(canvas), true); + } catch (Throwable err) { + // A desktop with no drag and drop service leaves the canvas as it was; the + // lightweight drag and drop is unaffected. + Log.e(err); + } + } + + /// Starts an AWT drag for the operation Codename One has decided on. Invoked on the + /// Codename One event dispatch thread while the mouse button is still down. + /// + /// #### Returns + /// + /// true when the export was handed to AWT; the outcome arrives later through + /// `NativeDragAndDrop#dragCompleted(int)` + static boolean startDrag(final JavaSEPort port, final NativeDragOperation op) { + if (op == null || GraphicsEnvironment.isHeadless()) { + return false; + } + final JavaSEPort.C target = port.dndGestureCanvas != null ? port.dndGestureCanvas : port.canvas; + if (target == null) { + return false; + } + final InputEvent trigger = port.dndLastInputEvent(); + if (!(trigger instanceof MouseEvent)) { + // AWT seeds a drag from the mouse event that provoked it and refuses without one. + return false; + } + final java.awt.Image dragImage = toAwtDragImage(op, target); + final Point offset = new Point( + (int) (op.getDragImageOffsetX() / target.canvasScale()), + (int) (op.getDragImageOffsetY() / target.canvasScale())); + exporting = op; + EventQueue.invokeLater(new Runnable() { + public void run() { + try { + TransferHandler handler = target.getTransferHandler(); + if (handler == null) { + NativeDragAndDrop.dragCompleted(NativeDragOperation.ACTION_NONE); + return; + } + if (dragImage != null) { + handler.setDragImage(dragImage); + handler.setDragImageOffset(offset); + } + handler.exportAsDrag(target, trigger, toAwtAction(preferred(op.getAllowedActions()))); + } catch (Throwable err) { + Log.e(err); + exporting = null; + NativeDragAndDrop.dragCompleted(NativeDragOperation.ACTION_NONE); + } + } + }); + return true; + } + + /// Forgets a prepared operation because the press turned out to be a click. + static void cancelDrag(JavaSEPort port) { + exporting = null; + } + + /// Renders the operation's drag image at the size AWT expects. + /// + /// Codename One images are in surface pixels while AWT places a drag image in points, so on + /// a scaled display the image has to come down by the backing scale or the user drags a + /// picture twice the size of the thing they grabbed. + private static java.awt.Image toAwtDragImage(NativeDragOperation op, JavaSEPort.C canvas) { + com.codename1.ui.Image image = op.getDragImage(); + if (image == null) { + return null; + } + Object peer = image.getImage(); + if (!(peer instanceof java.awt.Image)) { + return null; + } + java.awt.Image awt = (java.awt.Image) peer; + double scale = canvas.canvasScale(); + if (scale <= 1.0) { + return awt; + } + int w = Math.max(1, (int) (image.getWidth() / scale)); + int h = Math.max(1, (int) (image.getHeight() / scale)); + BufferedImage scaled = new BufferedImage(w, h, BufferedImage.TYPE_INT_ARGB); + java.awt.Graphics2D g = scaled.createGraphics(); + g.setRenderingHint(java.awt.RenderingHints.KEY_INTERPOLATION, + java.awt.RenderingHints.VALUE_INTERPOLATION_BILINEAR); + g.drawImage(awt, 0, 0, w, h, null); + g.dispose(); + return scaled; + } + + // ------------------------------------------------------------------------------------ + // Action mapping. AWT's constants are a different bit set from ours, and both sides use + // masks in some places and a single action in others. + // ------------------------------------------------------------------------------------ + + static int toAwtActions(int actions) { + int out = DnDConstants.ACTION_NONE; + if ((actions & NativeDragOperation.ACTION_COPY) != 0) { + out |= DnDConstants.ACTION_COPY; + } + if ((actions & NativeDragOperation.ACTION_MOVE) != 0) { + out |= DnDConstants.ACTION_MOVE; + } + if ((actions & NativeDragOperation.ACTION_LINK) != 0) { + out |= DnDConstants.ACTION_LINK; + } + return out; + } + + static int fromAwtActions(int actions) { + int out = NativeDragOperation.ACTION_NONE; + if ((actions & DnDConstants.ACTION_COPY) != 0) { + out |= NativeDragOperation.ACTION_COPY; + } + if ((actions & DnDConstants.ACTION_MOVE) != 0) { + out |= NativeDragOperation.ACTION_MOVE; + } + if ((actions & DnDConstants.ACTION_LINK) != 0) { + out |= NativeDragOperation.ACTION_LINK; + } + return out; + } + + static int toAwtAction(int action) { + return toAwtActions(action); + } + + static int preferred(int actions) { + if ((actions & NativeDragOperation.ACTION_COPY) != 0) { + return NativeDragOperation.ACTION_COPY; + } + if ((actions & NativeDragOperation.ACTION_MOVE) != 0) { + return NativeDragOperation.ACTION_MOVE; + } + if ((actions & NativeDragOperation.ACTION_LINK) != 0) { + return NativeDragOperation.ACTION_LINK; + } + return NativeDragOperation.ACTION_NONE; + } + + // ------------------------------------------------------------------------------------ + // Reading an AWT transferable as a ClipboardContent. + // ------------------------------------------------------------------------------------ + + /// Maps an AWT flavor onto the MIME type Codename One names that representation by, or null + /// when the flavor carries nothing the framework can express. + private static String mimeFor(DataFlavor flavor) { + if (flavor == null) { + return null; + } + if (DataFlavor.javaFileListFlavor.equals(flavor)) { + return ClipboardContent.MIME_FILE; + } + if (DataFlavor.imageFlavor.equals(flavor)) { + return ClipboardContent.MIME_PNG; + } + if (DataFlavor.stringFlavor.equals(flavor)) { + return ClipboardContent.MIME_TEXT; + } + String primary = flavor.getPrimaryType(); + String sub = flavor.getSubType(); + if (primary == null || sub == null) { + return null; + } + String mime = (primary + "/" + sub).toLowerCase(); + if ("application/rtf".equals(mime)) { + return ClipboardContent.MIME_RTF; + } + if ("application/x-java-file-list".equals(mime)) { + return ClipboardContent.MIME_FILE; + } + if (mime.startsWith("text/") || mime.startsWith("image/")) { + return mime; + } + return null; + } + + /// Describes a transferable as a `ClipboardContent`. + /// + /// #### Parameters + /// + /// - `transferable`: the AWT transferable + /// + /// - `flavors`: the flavors it is offering, in the order AWT reported them + /// + /// - `eager`: true to read every representation now, which is only correct inside a drop + /// callback; false to register providers that read on demand, which is what a drag in + /// progress needs + static ClipboardContent contentFor(final Transferable transferable, DataFlavor[] flavors, boolean eager) { + ClipboardContent content = new ClipboardContent(); + if (transferable == null || flavors == null) { + return content; + } + for (int iter = 0; iter < flavors.length; iter++) { + final DataFlavor flavor = flavors[iter]; + final String mime = mimeFor(flavor); + if (mime == null || content.hasMimeType(mime)) { + // The first flavor offering a MIME type wins: AWT lists them in the source's + // preference order, and the richer representation is the earlier one. + continue; + } + if (eager) { + Object value = readValue(transferable, flavor, mime); + if (value != null) { + content.setData(mime, value); + } + } else { + content.setDataProvider(mime, new ClipboardDataProvider() { + public Object getClipboardData(String requested) { + return readValue(transferable, flavor, requested); + } + }); + } + } + // A file list is also a URI list as far as most applications are concerned, and a drag + // out of a Linux file manager offers only the latter. Presenting both means a drop + // target that asks for files gets them either way. Declared the same way the rest of + // the content is -- eagerly on a drop, on demand during a drag -- so describing a drag + // still reads nothing. + if (!content.hasMimeType(ClipboardContent.MIME_FILE) && content.hasMimeType(ClipboardContent.MIME_URI_LIST)) { + if (eager) { + content.setFiles(pathsFromUriList(content.getText(ClipboardContent.MIME_URI_LIST))); + } else { + final ClipboardContent describing = content; + content.setDataProvider(ClipboardContent.MIME_FILE, new ClipboardDataProvider() { + public Object getClipboardData(String requested) { + String[] paths = pathsFromUriList(describing.getText(ClipboardContent.MIME_URI_LIST)); + if (paths == null) { + return null; + } + return paths.length == 1 ? (Object) paths[0] : paths; + } + }); + } + } + return content; + } + + /// Reads one representation out of a transferable, converting it into the value type the + /// MIME type implies. Returns null rather than throwing: a flavor that turns out to be + /// unreadable is simply one the drop does not offer. + private static Object readValue(Transferable transferable, DataFlavor flavor, String mime) { + try { + Object out = transferable.getTransferData(flavor); + if (out == null) { + return null; + } + if (ClipboardContent.MIME_FILE.equals(mime)) { + return filePaths(out); + } + if (ClipboardContent.MIME_PNG.equals(mime) && out instanceof java.awt.Image) { + return JavaSEPort.imageToPngBytes((java.awt.Image) out); + } + if (mime.startsWith("image/")) { + if (out instanceof byte[]) { + return out; + } + if (out instanceof InputStream) { + return readBytes((InputStream) out); + } + if (out instanceof java.awt.Image) { + return JavaSEPort.imageToPngBytes((java.awt.Image) out); + } + return null; + } + if (out instanceof byte[]) { + return out; + } + return JavaSEPort.clipboardText(out); + } catch (Throwable err) { + return null; + } + } + + /// Turns whatever a file flavor produced -- a list of files, or a URI list as text -- into + /// absolute paths. + private static Object filePaths(Object value) throws Exception { + if (value instanceof List) { + List list = (List) value; + List paths = new ArrayList(); + for (Object o : list) { + if (o instanceof File) { + paths.add(((File) o).getAbsolutePath()); + } else if (o != null) { + paths.add(o.toString()); + } + } + if (paths.isEmpty()) { + return null; + } + return paths.size() == 1 ? (Object) paths.get(0) : paths.toArray(new String[paths.size()]); + } + String[] fromUris = pathsFromUriList(JavaSEPort.clipboardText(value)); + if (fromUris == null) { + return null; + } + return fromUris.length == 1 ? (Object) fromUris[0] : fromUris; + } + + /// Parses the newline separated `text/uri-list` format, keeping only the `file:` entries, + /// which is what a drop from a file manager or the desktop consists of. + private static String[] pathsFromUriList(String uriList) { + if (uriList == null || uriList.length() == 0) { + return null; + } + List paths = new ArrayList(); + String[] lines = uriList.split("\r\n|\n|\r"); + for (int iter = 0; iter < lines.length; iter++) { + String line = lines[iter].trim(); + if (line.length() == 0 || line.charAt(0) == '#') { + continue; + } + try { + if (line.startsWith("file:")) { + paths.add(new File(new URI(line)).getAbsolutePath()); + } + } catch (Throwable err) { + // Not a URI this platform writes; skip it rather than fail the whole drop. + } + } + return paths.isEmpty() ? null : paths.toArray(new String[paths.size()]); + } + + private static byte[] readBytes(InputStream input) throws Exception { + try { + ByteArrayOutputStream out = new ByteArrayOutputStream(); + byte[] buffer = new byte[4096]; + int read; + while ((read = input.read(buffer)) >= 0) { + out.write(buffer, 0, read); + } + return out.toByteArray(); + } finally { + input.close(); + } + } + + // ------------------------------------------------------------------------------------ + + /// Exports whatever `#startDrag(JavaSEPort, com.codename1.ui.NativeDragOperation)` staged. + /// The payload travels as the same `JavaSEPort.RichTransferable` a copy uses, which is why + /// a drag out of the application lands correctly in a text editor, an image editor and a + /// file manager alike. + private static final class Cn1TransferHandler extends TransferHandler { + @Override + public int getSourceActions(JComponent c) { + NativeDragOperation op = exporting; + return op == null ? NONE : toAwtActions(op.getAllowedActions()); + } + + @Override + protected Transferable createTransferable(JComponent c) { + NativeDragOperation op = exporting; + return op == null ? null : new JavaSEPort.RichTransferable(op.getContent()); + } + + @Override + protected void exportDone(JComponent source, Transferable data, int action) { + exporting = null; + NativeDragAndDrop.dragCompleted(preferred(fromAwtActions(action))); + } + } + + /// Receives drags entering one canvas and routes them into the framework. + private static final class Cn1DropTargetListener implements DropTargetListener { + private final JavaSEPort.C canvas; + + Cn1DropTargetListener(JavaSEPort.C canvas) { + this.canvas = canvas; + } + + public void dragEnter(DropTargetDragEvent e) { + respond(e, true); + } + + public void dragOver(DropTargetDragEvent e) { + respond(e, false); + } + + public void dropActionChanged(DropTargetDragEvent e) { + respond(e, false); + } + + public void dragExit(DropTargetEvent e) { + try { + NativeDragAndDrop.dragExit(canvas.windowId); + } catch (Throwable err) { + Log.e(err); + } + } + + public void drop(DropTargetDropEvent e) { + try { + int allowed = fromAwtActions(e.getSourceActions()); + int action = preferred(fromAwtActions(e.getDropAction())); + if (action == NativeDragOperation.ACTION_NONE) { + action = preferred(allowed); + } + if (action == NativeDragOperation.ACTION_NONE) { + e.rejectDrop(); + return; + } + // Before reading anything: on every platform the transferable only becomes + // readable once the drop has been accepted, and it stops being readable when + // this method returns -- which is why the content is materialized here rather + // than handed to the event dispatch thread as a live view of the transfer. + e.acceptDrop(toAwtAction(action)); + ClipboardContent content = contentFor(e.getTransferable(), e.getCurrentDataFlavors(), true); + Point at = e.getLocation(); + int accepted = NativeDragAndDrop.drop(canvas.windowId, + canvas.scaleCoordinateX(at.x), canvas.scaleCoordinateY(at.y), + content, action); + e.dropComplete(accepted != NativeDragOperation.ACTION_NONE); + } catch (Throwable err) { + Log.e(err); + try { + e.dropComplete(false); + } catch (Throwable ignored) { + // The drop is already over; nothing left to report to. + } + } + } + + private void respond(DropTargetDragEvent e, boolean entering) { + try { + Point at = e.getLocation(); + int x = canvas.scaleCoordinateX(at.x); + int y = canvas.scaleCoordinateY(at.y); + ClipboardContent content = contentFor(e.getTransferable(), e.getCurrentDataFlavors(), false); + int allowed = fromAwtActions(e.getSourceActions()); + int action = entering + ? NativeDragAndDrop.dragEnter(canvas.windowId, x, y, content, allowed) + : NativeDragAndDrop.dragOver(canvas.windowId, x, y, content, allowed); + if (action == NativeDragOperation.ACTION_NONE) { + e.rejectDrag(); + } else { + e.acceptDrag(toAwtAction(action)); + } + } catch (Throwable err) { + Log.e(err); + try { + e.rejectDrag(); + } catch (Throwable ignored) { + // The drag has already moved on. + } + } + } + } +} diff --git a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSEPort.java b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSEPort.java index 2540198e600..fab4064a4e9 100644 --- a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSEPort.java +++ b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSEPort.java @@ -635,6 +635,15 @@ public Boolean isDarkMode() { private static boolean fullScreen; public float screenshotActualZoomLevel; private InputEvent lastInputEvent; + /// The canvas the gesture currently in progress started on, so a native drag can be + /// exported from the window the user is actually dragging in rather than the main one. + C dndGestureCanvas; + + /// The most recent mouse event, which is what `javax.swing.TransferHandler#exportAsDrag` + /// needs in order to seed a drag session. + InputEvent dndLastInputEvent() { + return lastInputEvent; + } public static double retinaScale = 1.0; static JMenuItem pause; @@ -1881,6 +1890,32 @@ public static void addFormChangeListener(com.codename1.ui.events.ActionListener formChangeListener.addListener(al); } + // ------------------------------------------------------------------------------------ + // Native drag and drop. The desktop is the one platform where a drag genuinely leaves the + // application -- onto another window, into a file manager, onto the desktop itself -- so + // this port implements the whole contract. See JavaSENativeDragAndDrop. + // ------------------------------------------------------------------------------------ + + @Override + public boolean isNativeDragAndDropSupported() { + return !java.awt.GraphicsEnvironment.isHeadless(); + } + + @Override + public boolean isNativeDragOutsideApplicationSupported() { + return isNativeDragAndDropSupported(); + } + + @Override + public boolean startNativeDrag(com.codename1.ui.NativeDragOperation op) { + return JavaSENativeDragAndDrop.startDrag(this, op); + } + + @Override + public void cancelNativeDrag() { + JavaSENativeDragAndDrop.cancelDrag(this); + } + @Override public void copyToClipboard(Object obj) { if (obj instanceof String || obj instanceof ClipboardContent) { @@ -1909,33 +1944,77 @@ public void run() { super.copyToClipboard(obj); //To change body of generated methods, choose Tools | Templates. } - private static final class RichTransferable implements Transferable { + /// Publishes a `ClipboardContent` to AWT, for a clipboard copy and for a native drag alike. + /// + /// The flavors are derived from the MIME types alone, never by reading the values. That is + /// deliberate: a drag may offer a representation registered through + /// `ClipboardContent#setDataProvider(java.lang.String, com.codename1.ui.ClipboardDataProvider)` + /// -- the file that is only written if the user actually drops on the desktop -- and reading + /// values here in order to decide what to advertise would build every one of them at the + /// moment the drag starts, which is exactly what the providers exist to avoid. + static final class RichTransferable implements Transferable { private final ClipboardContent data; private final DataFlavor[] flavors; RichTransferable(ClipboardContent data) { this.data = data; ArrayList available = new ArrayList(); - if (data.getText(ClipboardContent.MIME_TEXT) != null) { + String[] mimeTypes = data.getMimeTypes(); + boolean hasFiles = data.hasMimeType(ClipboardContent.MIME_FILE); + if (data.hasMimeType(ClipboardContent.MIME_TEXT)) { available.add(DataFlavor.stringFlavor); } - if (imageBytes(data) != null) { - available.add(DataFlavor.imageFlavor); - } - if (fileList(data) != null) { + if (hasFiles) { available.add(DataFlavor.javaFileListFlavor); + // GTK and a good deal of the web offer files this way and nothing else, so a + // drag that names files advertises both spellings of "these are files". + addTextFlavor(available, ClipboardContent.MIME_URI_LIST); } - String[] mimeTypes = data.getMimeTypes(); for (int i = 0; i < mimeTypes.length; i++) { - if (!ClipboardContent.MIME_TEXT.equals(mimeTypes[i]) - && !ClipboardContent.MIME_FILE.equals(mimeTypes[i]) - && data.getText(mimeTypes[i]) != null) { - available.add(new DataFlavor(mimeTypes[i] + ";class=java.lang.String", mimeTypes[i])); + String mime = mimeTypes[i]; + if (ClipboardContent.MIME_TEXT.equals(mime) || ClipboardContent.MIME_FILE.equals(mime)) { + continue; + } + if (mime.startsWith("image/")) { + if (!available.contains(DataFlavor.imageFlavor)) { + available.add(DataFlavor.imageFlavor); + } + addBinaryFlavor(available, mime); + } else if (mime.startsWith("text/")) { + addTextFlavor(available, mime); + } else { + addBinaryFlavor(available, mime); } } flavors = available.toArray(new DataFlavor[available.size()]); } + /// Adds a flavor whose representation class is `String`, which is how AWT carries text + /// of a specific MIME type -- `text/html` and `text/rtf` among them. + private static void addTextFlavor(ArrayList available, String mime) { + try { + DataFlavor flavor = new DataFlavor(mime + ";class=java.lang.String", mime); + if (!available.contains(flavor)) { + available.add(flavor); + } + } catch (Exception ex) { + // A MIME type AWT will not parse simply is not advertised. + } + } + + /// Adds a flavor whose representation is a stream of bytes, which is the only way to + /// hand an arbitrary binary payload -- a PDF, an archive -- to another application. + private static void addBinaryFlavor(ArrayList available, String mime) { + try { + DataFlavor flavor = new DataFlavor(mime + ";class=java.io.InputStream", mime); + if (!available.contains(flavor)) { + available.add(flavor); + } + } catch (Exception ex) { + // As above. + } + } + private static byte[] imageBytes(ClipboardContent data) { byte[] b = data.getBytes(ClipboardContent.MIME_PNG); if (b == null) { @@ -1950,16 +2029,8 @@ private static byte[] imageBytes(ClipboardContent data) { /// Resolves the `application/x-file-list` representation (a single path/URI `String` or a /// `String[]` of them) to AWT `File` objects for the native file-list clipboard flavor. private static java.util.List fileList(ClipboardContent data) { - Object value = data.getData(ClipboardContent.MIME_FILE); - if (value == null) { - return null; - } - String[] paths; - if (value instanceof String[]) { - paths = (String[]) value; - } else if (value instanceof String) { - paths = new String[] { (String) value }; - } else { + String[] paths = data.getFiles(); + if (paths == null) { return null; } java.util.List files = new java.util.ArrayList(); @@ -2000,20 +2071,20 @@ public boolean isDataFlavorSupported(DataFlavor flavor) { return false; } - public Object getTransferData(DataFlavor flavor) throws UnsupportedFlavorException { + public Object getTransferData(DataFlavor flavor) throws UnsupportedFlavorException, IOException { if (DataFlavor.stringFlavor.equals(flavor)) { - return data.getText(ClipboardContent.MIME_TEXT); + String text = data.getText(ClipboardContent.MIME_TEXT); + if (text != null) { + return text; + } + throw new UnsupportedFlavorException(flavor); } if (DataFlavor.imageFlavor.equals(flavor)) { byte[] bytes = imageBytes(data); if (bytes != null) { - try { - java.awt.Image img = ImageIO.read(new ByteArrayInputStream(bytes)); - if (img != null) { - return img; - } - } catch (IOException ex) { - // fall through to unsupported + java.awt.Image img = ImageIO.read(new ByteArrayInputStream(bytes)); + if (img != null) { + return img; } } throw new UnsupportedFlavorException(flavor); @@ -2025,10 +2096,33 @@ public Object getTransferData(DataFlavor flavor) throws UnsupportedFlavorExcepti } throw new UnsupportedFlavorException(flavor); } - String value = data.getText(flavor.getPrimaryType() + "/" + flavor.getSubType()); - if (value != null) { + String mime = (flavor.getPrimaryType() + "/" + flavor.getSubType()).toLowerCase(); + if (ClipboardContent.MIME_URI_LIST.equals(mime) && !data.hasMimeType(ClipboardContent.MIME_URI_LIST)) { + // Synthesized from the file list rather than stored, so a source only has to + // name its files once. + java.util.List files = fileList(data); + if (files != null) { + StringBuilder uris = new StringBuilder(); + for (java.io.File f : files) { + uris.append(f.toURI().toString()).append("\r\n"); + } + return uris.toString(); + } + throw new UnsupportedFlavorException(flavor); + } + Object value = data.getData(mime); + if (value instanceof String) { + if (InputStream.class.equals(flavor.getRepresentationClass())) { + return new ByteArrayInputStream(((String) value).getBytes("UTF-8")); + } return value; } + if (value instanceof byte[]) { + if (InputStream.class.equals(flavor.getRepresentationClass())) { + return new ByteArrayInputStream((byte[]) value); + } + return new String((byte[]) value, "UTF-8"); + } throw new UnsupportedFlavorException(flavor); } } @@ -2104,7 +2198,7 @@ public Object getPasteDataFromClipboard() { /// Encodes an AWT clipboard image as PNG bytes so it can travel through the CN1 clipboard as an /// {@code image/png} representation. - private static byte[] imageToPngBytes(java.awt.Image image) { + static byte[] imageToPngBytes(java.awt.Image image) { try { BufferedImage buffered; if (image instanceof BufferedImage) { @@ -2127,7 +2221,7 @@ private static byte[] imageToPngBytes(java.awt.Image image) { } } - private static String clipboardText(Object value) throws IOException { + static String clipboardText(Object value) throws IOException { if (value instanceof String) { return (String)value; } @@ -3109,6 +3203,9 @@ int surfaceHeight() { addHierarchyBoundsListener(this); setFocusable(true); setOpaque(false); + // Native drag and drop, both directions: this canvas becomes an AWT drop target + // and gets a transfer handler that can export a Codename One drag to the desktop. + JavaSENativeDragAndDrop.install(this); installNativeMagnificationListeners(); addHierarchyListener(new HierarchyListener() { public void hierarchyChanged(HierarchyEvent e) { @@ -4044,7 +4141,7 @@ public void actionPerformed(ActionEvent e) { return true; } - private int scaleCoordinateX(int coordinate) { + int scaleCoordinateX(int coordinate) { if (getScreenCoordinates() != null) { return (int) (retinaScale * coordinate / zoomLevel - (getScreenCoordinates().x + x)); } @@ -4053,7 +4150,7 @@ private int scaleCoordinateX(int coordinate) { return (int)(coordinate * canvasScale()); } - private int scaleCoordinateY(int coordinate) { + int scaleCoordinateY(int coordinate) { if (getScreenCoordinates() != null) { return (int) (retinaScale * coordinate / zoomLevel - (getScreenCoordinates().y + y)); } @@ -4071,6 +4168,7 @@ public void mousePressed(MouseEvent e) { } } this.mouseDown = true; + JavaSEPort.this.dndGestureCanvas = this; com.codename1.ui.TopLevelContainer f = canvasTopLevel(); if (f != null) { int x = scaleCoordinateX(e.getX()); diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.h b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h new file mode 100644 index 00000000000..9c0d000a1ed --- /dev/null +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h @@ -0,0 +1,102 @@ +/* + * Copyright (c) 2012, Codename One and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. Codename One designates this + * particular file as subject to the "Classpath" exception as provided + * by Oracle in the LICENSE file that accompanied this code. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Codename One through http://www.codenameone.com/ if you + * need additional information or have any questions. + */ + +#ifndef CN1DragAndDrop_h +#define CN1DragAndDrop_h + +#import +#include "TargetConditionals.h" +#import "CN1AppleUI.h" + +/* + * Native drag and drop for the UIKit ports: iOS, iPadOS and Mac Catalyst. + * + * UIKit owns the drag gesture. UIDragInteraction has its own recognizer -- a long press on a + * phone, a lift on a trackpad -- and asks the delegate what is being dragged at the moment it + * fires. So the framework cannot start a session on its own drag threshold the way the desktop + * port does; instead it stages the operation on the press (CN1PrepareNativeDrag) and this file + * calls back into Java when UIKit decides a drag has begun. + * + * The payload itself is fetched at that later moment rather than on the press, because a drag + * that may carry a file the application has not written yet must not write it every time the + * user merely touches the component. + * + * watchOS, tvOS and the native AppKit port have no UIDragInteraction; there the whole file + * compiles to the unsupported answers below and the framework keeps its lightweight in-form + * drag and drop. + */ + +/// True when this build can drag through the operating system at all. +BOOL CN1DragAndDropSupported(void); + +/// True when a drag started here can be dropped outside the application. On iPadOS and Mac +/// Catalyst it can; on iPhone a drag stays inside the application, because there is no second +/// application on screen to drop it into. +BOOL CN1DragOutsideAppSupported(void); + +/// Attaches the drag and the drop interactions to the Codename One surface. +void CN1InstallDragAndDrop(CN1View* view); + +/// Stages the drag a press has made possible: which representations it can offer, what the +/// receiver may do with them, and the image to show under the finger. The representations are +/// named but not built -- CN1SetNativeDragPayload delivers the bytes once the drag really +/// starts. +/// +/// mimeTypes is newline separated. +void CN1PrepareNativeDrag(NSString* mimeTypes, int allowedActions, NSData* dragImagePng, + int touchX, int touchY); + +/// Delivers the payload for the session UIKit has just started. Called from Java, from inside +/// the session-started callback below. +/// +/// fileUris is newline separated and may be nil. +void CN1SetNativeDragPayload(NSString* plain, NSString* html, NSString* rtf, + NSData* image, NSString* fileUris); + +/// Drops whatever CN1PrepareNativeDrag staged, because the press turned out to be a tap. +void CN1CancelNativeDrag(void); + +/* + * The Java side of the bridge. Defined in IOSNative.m, where every piece of ParparVM thread + * state handling lives, so that this file stays plain UIKit. + */ + +/// Reports a drag moving over the surface and returns the action a drop would perform, or 0. +int CN1NativeDragDeliverOver(int x, int y, NSString* mimeTypes, int allowedActions, BOOL entering); + +/// Reports a drag leaving the surface. +void CN1NativeDragDeliverExit(void); + +/// Delivers a drop and returns the action actually accepted, or 0 when nothing took it. +int CN1NativeDragDeliverDrop(int x, int y, NSString* plain, NSString* html, NSString* rtf, + NSData* image, NSString* fileUris, int action); + +/// Announces that UIKit has started a drag session. Returns the actions the framework's staged +/// operation allows, or 0 when it has none -- in which case no drag begins. The Java side calls +/// CN1SetNativeDragPayload from inside this call. +int CN1NativeDragDeliverSessionStarted(void); + +/// Reports the outcome of a session this application started. +void CN1NativeDragDeliverCompleted(int action); + +#endif diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m new file mode 100644 index 00000000000..0631a43eba6 --- /dev/null +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -0,0 +1,521 @@ +/* + * Copyright (c) 2012, Codename One and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. Codename One designates this + * particular file as subject to the "Classpath" exception as provided + * by Oracle in the LICENSE file that accompanied this code. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Codename One through http://www.codenameone.com/ if you + * need additional information or have any questions. + */ + +#import "CN1DragAndDrop.h" + +#define CN1_DND_ACTION_NONE 0 +#define CN1_DND_ACTION_COPY 1 +#define CN1_DND_ACTION_MOVE 2 +#define CN1_DND_ACTION_LINK 4 + +#if TARGET_OS_OSX || TARGET_OS_WATCH || TARGET_OS_TV + +// No UIDragInteraction on these targets. The framework asks first and falls back to its +// lightweight in-form drag and drop, so these answers are the whole implementation. + +BOOL CN1DragAndDropSupported(void) { + return NO; +} + +BOOL CN1DragOutsideAppSupported(void) { + return NO; +} + +void CN1InstallDragAndDrop(CN1View* view) { +} + +void CN1PrepareNativeDrag(NSString* mimeTypes, int allowedActions, NSData* dragImagePng, + int touchX, int touchY) { +} + +void CN1SetNativeDragPayload(NSString* plain, NSString* html, NSString* rtf, + NSData* image, NSString* fileUris) { +} + +void CN1CancelNativeDrag(void) { +} + +#else + +/// The device pixel scale, owned by the view controller. UIKit reports drag positions in +/// points and the framework works in pixels, exactly as the touch path does. +extern float scaleValue; + +/// What the press staged: the representations the drag could offer, what a receiver may do +/// with them, and the preview. Named but not built -- see CN1DragAndDrop.h. +static NSArray* cn1PreparedMimes = nil; +static int cn1PreparedActions = CN1_DND_ACTION_NONE; +static UIImage* cn1PreparedPreview = nil; +static CGPoint cn1PreparedTouch; + +/// The payload of the session UIKit is currently running, delivered by +/// CN1SetNativeDragPayload once the drag has actually begun. Keyed by uniform type identifier +/// so the item provider can register each one directly. +static NSMutableDictionary* cn1DragData = nil; +static NSArray* cn1DragFileUrls = nil; + +/// The last action the framework agreed to, reused when a drop arrives without one. +static int cn1LastDropAction = CN1_DND_ACTION_NONE; + +/// True while this application is the source of the session in progress. +static BOOL cn1DraggingOut = NO; + +/// The MIME types the framework names, mapped onto the uniform type identifiers UIKit and +/// every other application on the system speak. +static NSString* cn1UtiForMime(NSString* mime) { + if ([mime isEqualToString:@"text/plain"]) { + return @"public.utf8-plain-text"; + } + if ([mime isEqualToString:@"text/html"]) { + return @"public.html"; + } + if ([mime isEqualToString:@"text/rtf"]) { + return @"public.rtf"; + } + if ([mime isEqualToString:@"text/markdown"]) { + return @"net.daringfireball.markdown"; + } + if ([mime isEqualToString:@"image/png"]) { + return @"public.png"; + } + if ([mime isEqualToString:@"image/jpeg"]) { + return @"public.jpeg"; + } + if ([mime isEqualToString:@"image/gif"]) { + return @"com.compuserve.gif"; + } + if ([mime isEqualToString:@"application/x-file-list"]) { + return @"public.file-url"; + } + if ([mime isEqualToString:@"text/uri-list"]) { + return @"public.url"; + } + return nil; +} + +static NSString* cn1MimeForUti(NSString* uti) { + if ([uti isEqualToString:@"public.utf8-plain-text"] || [uti isEqualToString:@"public.plain-text"] + || [uti isEqualToString:@"public.text"]) { + return @"text/plain"; + } + if ([uti isEqualToString:@"public.html"]) { + return @"text/html"; + } + if ([uti isEqualToString:@"public.rtf"]) { + return @"text/rtf"; + } + if ([uti isEqualToString:@"net.daringfireball.markdown"]) { + return @"text/markdown"; + } + if ([uti isEqualToString:@"public.png"]) { + return @"image/png"; + } + if ([uti isEqualToString:@"public.jpeg"]) { + return @"image/jpeg"; + } + if ([uti isEqualToString:@"com.compuserve.gif"]) { + return @"image/gif"; + } + if ([uti isEqualToString:@"public.file-url"]) { + return @"application/x-file-list"; + } + if ([uti isEqualToString:@"public.url"]) { + return @"text/uri-list"; + } + return nil; +} + +/// Files one representation under the uniform type identifier the rest of the system knows it +/// by, so the mapping lives in cn1UtiForMime rather than being spelled out again here. +static void cn1PutRepresentation(NSMutableDictionary* data, NSString* mime, NSData* value) { + if (value == nil || value.length == 0) { + return; + } + NSString* uti = cn1UtiForMime(mime); + if (uti != nil) { + [data setObject:value forKey:uti]; + } +} + +static UIDropOperation cn1DropOperationFor(int action) { + if ((action & CN1_DND_ACTION_MOVE) != 0) { + return UIDropOperationMove; + } + if ((action & CN1_DND_ACTION_LINK) != 0) { + // UIKit has no "link"; a receiver that wanted one still wants the transfer to happen, + // and copy is the operation that says so without claiming the source loses its data. + return UIDropOperationCopy; + } + if ((action & CN1_DND_ACTION_COPY) != 0) { + return UIDropOperationCopy; + } + return UIDropOperationCancel; +} + +/// The MIME types a drag in progress is offering, newline separated, derived from the type +/// identifiers alone. UIKit does not hand over the data until the drop and the framework only +/// needs the names in order to decide whether any component wants the drag. +static NSString* cn1MimesForSession(id session) { + NSMutableArray* mimes = [NSMutableArray array]; + for (UIDragItem* item in session.items) { + for (NSString* uti in item.itemProvider.registeredTypeIdentifiers) { + NSString* mime = cn1MimeForUti(uti); + if (mime != nil && ![mimes containsObject:mime]) { + [mimes addObject:mime]; + } + } + // A provider that can vend a file is a file drag whatever else it also offers, which is + // how a document dragged out of Files reaches a target that asked for files. + if ([item.itemProvider hasItemConformingToTypeIdentifier:@"public.file-url"] + && ![mimes containsObject:@"application/x-file-list"]) { + [mimes addObject:@"application/x-file-list"]; + } + } + return [mimes componentsJoinedByString:@"\n"]; +} + +BOOL CN1DragAndDropSupported(void) { + if (@available(iOS 11.0, *)) { + return YES; + } + return NO; +} + +BOOL CN1DragOutsideAppSupported(void) { + if (@available(iOS 11.0, *)) { +#if TARGET_OS_MACCATALYST + return YES; +#else + // UIDragInteraction only carries a drag out of the application where the system has + // somewhere to carry it to: an iPad, or an iPhone running iPadOS style multitasking. + // On a phone in full screen the same session works, but it can only end on one of this + // application's own components. + return [UIDevice currentDevice].userInterfaceIdiom == UIUserInterfaceIdiomPad; +#endif + } + return NO; +} + +void CN1PrepareNativeDrag(NSString* mimeTypes, int allowedActions, NSData* dragImagePng, + int touchX, int touchY) { + NSArray* mimes = mimeTypes == nil || mimeTypes.length == 0 + ? nil : [mimeTypes componentsSeparatedByString:@"\n"]; +#ifndef CN1_USE_ARC + [cn1PreparedMimes release]; + [cn1PreparedPreview release]; +#endif + cn1PreparedMimes = mimes; + cn1PreparedActions = allowedActions; + cn1PreparedPreview = dragImagePng == nil ? nil : [UIImage imageWithData:dragImagePng]; + cn1PreparedTouch = CGPointMake(touchX / scaleValue, touchY / scaleValue); +#ifndef CN1_USE_ARC + [cn1PreparedMimes retain]; + [cn1PreparedPreview retain]; +#endif +} + +void CN1SetNativeDragPayload(NSString* plain, NSString* html, NSString* rtf, + NSData* image, NSString* fileUris) { + NSMutableDictionary* data = [NSMutableDictionary dictionary]; + cn1PutRepresentation(data, @"text/plain", plain == nil ? nil : [plain dataUsingEncoding:NSUTF8StringEncoding]); + cn1PutRepresentation(data, @"text/html", html == nil ? nil : [html dataUsingEncoding:NSUTF8StringEncoding]); + cn1PutRepresentation(data, @"text/rtf", rtf == nil ? nil : [rtf dataUsingEncoding:NSUTF8StringEncoding]); + cn1PutRepresentation(data, @"image/png", image); + NSMutableArray* urls = [NSMutableArray array]; + if (fileUris != nil && fileUris.length > 0) { + for (NSString* entry in [fileUris componentsSeparatedByString:@"\n"]) { + if (entry.length == 0) { + continue; + } + // ClipboardContent's file representation permits a raw local path as well as a + // file: URI, and URLWithString: turns a path into a scheme-less relative URL that + // no receiver can open. + NSURL* url = ([entry hasPrefix:@"/"] || [entry hasPrefix:@"~"]) + ? [NSURL fileURLWithPath:[entry stringByExpandingTildeInPath]] + : [NSURL URLWithString:entry]; + if (url != nil) { + [urls addObject:url]; + } + } + } +#ifndef CN1_USE_ARC + [cn1DragData release]; + [cn1DragFileUrls release]; +#endif + cn1DragData = data; + cn1DragFileUrls = urls; +#ifndef CN1_USE_ARC + [cn1DragData retain]; + [cn1DragFileUrls retain]; +#endif +} + +void CN1CancelNativeDrag(void) { +#ifndef CN1_USE_ARC + [cn1PreparedMimes release]; + [cn1PreparedPreview release]; +#endif + cn1PreparedMimes = nil; + cn1PreparedPreview = nil; + cn1PreparedActions = CN1_DND_ACTION_NONE; +} + +API_AVAILABLE(ios(11.0)) +@interface CN1DragAndDropDelegate : NSObject +@end + +@implementation CN1DragAndDropDelegate + +// ---- the drag out half ------------------------------------------------------------------ + +- (NSArray *)dragInteraction:(UIDragInteraction *)interaction + itemsForBeginningSession:(id)session { + if (cn1PreparedMimes == nil || cn1PreparedMimes.count == 0) { + return @[]; + } + // Asking the framework now, rather than on the press, is what lets a promised file stay + // unwritten until a drag really happens. The Java side fills cn1DragData from inside this + // call through CN1SetNativeDragPayload. + int allowed = CN1NativeDragDeliverSessionStarted(); + if (allowed == CN1_DND_ACTION_NONE) { + return @[]; + } + cn1DraggingOut = YES; + + NSMutableArray* items = [NSMutableArray array]; + // Files first, one item each: a receiver that copies documents expects one item per + // document, and collapsing several into one loses all but the first. + for (NSURL* url in cn1DragFileUrls) { + NSItemProvider* provider = [[NSItemProvider alloc] initWithContentsOfURL:url]; + if (provider == nil) { + continue; + } + UIDragItem* item = [[UIDragItem alloc] initWithItemProvider:provider]; + [items addObject:item]; +#ifndef CN1_USE_ARC + [provider release]; + [item release]; +#endif + } + if (cn1DragData.count > 0) { + NSItemProvider* provider = [[NSItemProvider alloc] init]; + for (NSString* uti in cn1DragData) { + NSData* payload = [cn1DragData objectForKey:uti]; +#ifndef CN1_USE_ARC + [payload retain]; +#endif + [provider registerDataRepresentationForTypeIdentifier:uti + visibility:NSItemProviderRepresentationVisibilityAll + loadHandler:^NSProgress *(void (^completion)(NSData *, NSError *)) { + completion(payload, nil); + return nil; + }]; + } + UIDragItem* item = [[UIDragItem alloc] initWithItemProvider:provider]; + [items addObject:item]; +#ifndef CN1_USE_ARC + [provider release]; + [item release]; +#endif + } + if (items.count == 0) { + cn1DraggingOut = NO; + CN1NativeDragDeliverCompleted(CN1_DND_ACTION_NONE); + } + return items; +} + +- (UIDragPreview *)dragInteraction:(UIDragInteraction *)interaction + previewForLiftingItem:(UIDragItem *)item + session:(id)session { + if (cn1PreparedPreview == nil) { + // Without one UIKit snapshots the interaction's view, which is the whole Codename One + // surface; nil here leaves UIKit to its default rather than dragging the entire screen. + return nil; + } + UIImageView* view = [[UIImageView alloc] initWithImage:cn1PreparedPreview]; + UIDragPreview* preview = [[UIDragPreview alloc] initWithView:view]; +#ifndef CN1_USE_ARC + [view release]; + [preview autorelease]; +#endif + return preview; +} + +- (void)dragInteraction:(UIDragInteraction *)interaction + session:(id)session + didEndWithOperation:(UIDropOperation)operation { + cn1DraggingOut = NO; + int action = CN1_DND_ACTION_NONE; + if (operation == UIDropOperationCopy) { + action = CN1_DND_ACTION_COPY; + } else if (operation == UIDropOperationMove) { + action = CN1_DND_ACTION_MOVE; + } + CN1NativeDragDeliverCompleted(action); +} + +// ---- the drop half ---------------------------------------------------------------------- + +- (BOOL)dropInteraction:(UIDropInteraction *)interaction canHandleSession:(id)session { + return YES; +} + +- (void)dropInteraction:(UIDropInteraction *)interaction sessionDidEnter:(id)session { + CGPoint point = [session locationInView:interaction.view]; + cn1LastDropAction = CN1NativeDragDeliverOver((int)(point.x * scaleValue), (int)(point.y * scaleValue), + cn1MimesForSession(session), CN1_DND_ACTION_COPY, YES); +} + +- (UIDropProposal *)dropInteraction:(UIDropInteraction *)interaction + sessionDidUpdate:(id)session { + CGPoint point = [session locationInView:interaction.view]; + cn1LastDropAction = CN1NativeDragDeliverOver((int)(point.x * scaleValue), (int)(point.y * scaleValue), + cn1MimesForSession(session), CN1_DND_ACTION_COPY, NO); + UIDropProposal* proposal = [[UIDropProposal alloc] initWithDropOperation:cn1DropOperationFor(cn1LastDropAction)]; +#ifndef CN1_USE_ARC + [proposal autorelease]; +#endif + return proposal; +} + +- (void)dropInteraction:(UIDropInteraction *)interaction sessionDidExit:(id)session { + cn1LastDropAction = CN1_DND_ACTION_NONE; + CN1NativeDragDeliverExit(); +} + +- (void)dropInteraction:(UIDropInteraction *)interaction performDrop:(id)session { + CGPoint point = [session locationInView:interaction.view]; + const int x = (int)(point.x * scaleValue); + const int y = (int)(point.y * scaleValue); + const int action = cn1LastDropAction == CN1_DND_ACTION_NONE ? CN1_DND_ACTION_COPY : cn1LastDropAction; + + // Every representation is loaded asynchronously and independently, so the framework is only + // told about the drop once they have all answered. Delivering per representation instead + // would give the application several drops for one gesture, each missing the others. + NSMutableDictionary* collected = [[NSMutableDictionary alloc] init]; + NSMutableArray* files = [[NSMutableArray alloc] init]; + dispatch_group_t group = dispatch_group_create(); + + for (UIDragItem* item in session.items) { + NSItemProvider* provider = item.itemProvider; + if ([provider hasItemConformingToTypeIdentifier:@"public.file-url"]) { + dispatch_group_enter(group); + [provider loadFileRepresentationForTypeIdentifier:@"public.file-url" + completionHandler:^(NSURL* url, NSError* error) { + if (url != nil) { + // The URL is only valid inside this handler, so the file is copied out + // before it is named to the application. A path handed over without + // copying is unreadable by the time the event dispatch thread sees it. + NSString* name = url.lastPathComponent; + if (name == nil || name.length == 0) { + name = @"dropped"; + } + NSString* target = [NSTemporaryDirectory() stringByAppendingPathComponent: + [NSString stringWithFormat:@"cn1-drop-%@-%@", + [[NSUUID UUID] UUIDString], name]]; + NSError* copyError = nil; + if ([[NSFileManager defaultManager] copyItemAtURL:url + toURL:[NSURL fileURLWithPath:target] + error:©Error]) { + @synchronized (files) { + [files addObject:target]; + } + } + } + dispatch_group_leave(group); + }]; + continue; + } + for (NSString* uti in provider.registeredTypeIdentifiers) { + NSString* mime = cn1MimeForUti(uti); + if (mime == nil) { + continue; + } + dispatch_group_enter(group); + [provider loadDataRepresentationForTypeIdentifier:uti + completionHandler:^(NSData* data, NSError* error) { + if (data != nil) { + @synchronized (collected) { + if ([collected objectForKey:mime] == nil) { + [collected setObject:data forKey:mime]; + } + } + } + dispatch_group_leave(group); + }]; + } + } + + dispatch_group_notify(group, dispatch_get_main_queue(), ^{ + NSData* plainData = [collected objectForKey:@"text/plain"]; + NSData* htmlData = [collected objectForKey:@"text/html"]; + NSData* rtfData = [collected objectForKey:@"text/rtf"]; + NSData* imageData = [collected objectForKey:@"image/png"]; + if (imageData == nil) { + imageData = [collected objectForKey:@"image/jpeg"]; + } + NSString* plain = plainData == nil ? nil + : [[[NSString alloc] initWithData:plainData encoding:NSUTF8StringEncoding] autorelease]; + NSString* html = htmlData == nil ? nil + : [[[NSString alloc] initWithData:htmlData encoding:NSUTF8StringEncoding] autorelease]; + NSString* rtf = rtfData == nil ? nil + : [[[NSString alloc] initWithData:rtfData encoding:NSUTF8StringEncoding] autorelease]; + NSString* fileUris = files.count == 0 ? nil : [files componentsJoinedByString:@"\n"]; + CN1NativeDragDeliverDrop(x, y, plain, html, rtf, imageData, fileUris, action); + cn1LastDropAction = CN1_DND_ACTION_NONE; +#ifndef CN1_USE_ARC + [collected release]; + [files release]; +#endif + }); +#ifndef CN1_USE_ARC + dispatch_release(group); +#endif +} + +@end + +void CN1InstallDragAndDrop(CN1View* view) { + if (view == nil || !CN1DragAndDropSupported()) { + return; + } + if (@available(iOS 11.0, *)) { + CN1DragAndDropDelegate* delegate = [[CN1DragAndDropDelegate alloc] init]; + UIDragInteraction* drag = [[UIDragInteraction alloc] initWithDelegate:delegate]; + // Without this a drag never begins on iPhone: UIKit enables drag interactions on iPad + // by default and leaves them off elsewhere. + drag.enabled = YES; + [view addInteraction:drag]; + UIDropInteraction* drop = [[UIDropInteraction alloc] initWithDelegate:delegate]; + [view addInteraction:drop]; + // The delegate is deliberately not released: the interactions hold their delegate + // weakly and it has to outlive the surface, which lives for the life of the process. +#ifndef CN1_USE_ARC + [drag release]; + [drop release]; +#endif + } +} + +#endif diff --git a/Ports/iOSPort/nativeSources/CodenameOne_GLViewController.m b/Ports/iOSPort/nativeSources/CodenameOne_GLViewController.m index f5123cc0edb..cd33c184b43 100644 --- a/Ports/iOSPort/nativeSources/CodenameOne_GLViewController.m +++ b/Ports/iOSPort/nativeSources/CodenameOne_GLViewController.m @@ -43,6 +43,7 @@ #import "METALView.h" #import "CN1Metalcompat.h" #endif +#import "CN1DragAndDrop.h" #import "ExecutableOp.h" #import "FillRect.h" #import "ClipRect.h" @@ -3262,6 +3263,8 @@ - (void)viewDidLoad { [self cn1InstallHoverRecognizer]; [self cn1InstallScrollRecognizer]; [self cn1InstallPinchRecognizer]; + // Native drag and drop, both directions. Inert where the platform has no drag interaction. + CN1InstallDragAndDrop(self.view); [self cn1InstallRotationRecognizer]; //replaceViewDidLoad [self initGoogleConnect]; @@ -3283,6 +3286,8 @@ - (void)viewDidLoad { [self cn1InstallHoverRecognizer]; [self cn1InstallScrollRecognizer]; [self cn1InstallPinchRecognizer]; + // Native drag and drop, both directions. Inert where the platform has no drag interaction. + CN1InstallDragAndDrop(self.view); [self cn1InstallRotationRecognizer]; //replaceViewDidLoad [self initGoogleConnect]; diff --git a/Ports/iOSPort/nativeSources/IOSNative.m b/Ports/iOSPort/nativeSources/IOSNative.m index 17165743dec..e3df8646057 100644 --- a/Ports/iOSPort/nativeSources/IOSNative.m +++ b/Ports/iOSPort/nativeSources/IOSNative.m @@ -53,6 +53,7 @@ #endif #import "CN1AudioUnit.h" #import "CN1AppleUI.h" +#import "CN1DragAndDrop.h" #if TARGET_OS_OSX /* @@ -1054,6 +1055,83 @@ JAVA_OBJECT com_codename1_impl_ios_IOSNative_getClipboardContent___java_lang_Str extern NSData* arrayToData(JAVA_OBJECT arr); extern JAVA_OBJECT nsDataToByteArr(NSData *data); +/* + * Native drag and drop. The UIKit half lives in CN1DragAndDrop.m; everything here is the + * ParparVM boundary, kept in this file with the other bridges so all the thread-state handling + * stays in one place. + */ + +JAVA_BOOLEAN com_codename1_impl_ios_IOSNative_isNativeDragAndDropSupported___R_boolean(CN1_THREAD_STATE_MULTI_ARG JAVA_OBJECT instanceObject) { + return CN1DragAndDropSupported() ? JAVA_TRUE : JAVA_FALSE; +} + +JAVA_BOOLEAN com_codename1_impl_ios_IOSNative_isNativeDragOutsideAppSupported___R_boolean(CN1_THREAD_STATE_MULTI_ARG JAVA_OBJECT instanceObject) { + return CN1DragOutsideAppSupported() ? JAVA_TRUE : JAVA_FALSE; +} + +void com_codename1_impl_ios_IOSNative_prepareNativeDrag___java_lang_String_int_byte_1ARRAY_int_int(CN1_THREAD_STATE_MULTI_ARG JAVA_OBJECT instanceObject, JAVA_OBJECT mimeTypes, JAVA_INT allowedActions, JAVA_OBJECT dragImagePng, JAVA_INT touchX, JAVA_INT touchY) { + POOL_BEGIN(); + NSString* mimes = mimeTypes == JAVA_NULL ? nil : toNSString(CN1_THREAD_STATE_PASS_ARG mimeTypes); + NSData* preview = dragImagePng == JAVA_NULL ? nil : arrayToData(dragImagePng); + // On the main thread: the interactions and the state they read live there, and this is + // called from the event dispatch thread as the press is dispatched. + dispatch_async(dispatch_get_main_queue(), ^{ + CN1PrepareNativeDrag(mimes, (int)allowedActions, preview, (int)touchX, (int)touchY); + }); + POOL_END(); +} + +void com_codename1_impl_ios_IOSNative_setNativeDragPayload___java_lang_String_java_lang_String_java_lang_String_byte_1ARRAY_java_lang_String(CN1_THREAD_STATE_MULTI_ARG JAVA_OBJECT instanceObject, JAVA_OBJECT plain, JAVA_OBJECT html, JAVA_OBJECT rtf, JAVA_OBJECT image, JAVA_OBJECT fileUris) { + POOL_BEGIN(); + // Synchronously, unlike prepare: this runs inside the session-started callback on the main + // thread and the item providers are built from it the moment it returns. + CN1SetNativeDragPayload(plain == JAVA_NULL ? nil : toNSString(CN1_THREAD_STATE_PASS_ARG plain), + html == JAVA_NULL ? nil : toNSString(CN1_THREAD_STATE_PASS_ARG html), + rtf == JAVA_NULL ? nil : toNSString(CN1_THREAD_STATE_PASS_ARG rtf), + image == JAVA_NULL ? nil : arrayToData(image), + fileUris == JAVA_NULL ? nil : toNSString(CN1_THREAD_STATE_PASS_ARG fileUris)); + POOL_END(); +} + +void com_codename1_impl_ios_IOSNative_cancelNativeDrag__(CN1_THREAD_STATE_MULTI_ARG JAVA_OBJECT instanceObject) { + dispatch_async(dispatch_get_main_queue(), ^{ + CN1CancelNativeDrag(); + }); +} + +int CN1NativeDragDeliverOver(int x, int y, NSString* mimeTypes, int allowedActions, BOOL entering) { + return (int)com_codename1_impl_ios_IOSImplementation_nativeDragOverCallback___int_int_java_lang_String_int_boolean_R_int( + CN1_THREAD_GET_STATE_PASS_ARG x, y, + fromNSString(CN1_THREAD_GET_STATE_PASS_ARG mimeTypes), + allowedActions, entering ? JAVA_TRUE : JAVA_FALSE); +} + +void CN1NativeDragDeliverExit(void) { + com_codename1_impl_ios_IOSImplementation_nativeDragExitCallback__(CN1_THREAD_GET_STATE_PASS_SINGLE_ARG); +} + +int CN1NativeDragDeliverDrop(int x, int y, NSString* plain, NSString* html, NSString* rtf, + NSData* image, NSString* fileUris, int action) { + return (int)com_codename1_impl_ios_IOSImplementation_nativeDropCallback___int_int_java_lang_String_java_lang_String_java_lang_String_byte_1ARRAY_java_lang_String_int_R_int( + CN1_THREAD_GET_STATE_PASS_ARG x, y, + fromNSString(CN1_THREAD_GET_STATE_PASS_ARG plain), + fromNSString(CN1_THREAD_GET_STATE_PASS_ARG html), + fromNSString(CN1_THREAD_GET_STATE_PASS_ARG rtf), + image == nil ? JAVA_NULL : nsDataToByteArr(image), + fromNSString(CN1_THREAD_GET_STATE_PASS_ARG fileUris), + action); +} + +int CN1NativeDragDeliverSessionStarted(void) { + return (int)com_codename1_impl_ios_IOSImplementation_nativeDragSessionStartedCallback___R_int( + CN1_THREAD_GET_STATE_PASS_SINGLE_ARG); +} + +void CN1NativeDragDeliverCompleted(int action) { + com_codename1_impl_ios_IOSImplementation_nativeDragCompletedCallback___int( + CN1_THREAD_GET_STATE_PASS_ARG action); +} + #if TARGET_OS_OSX /// The pasteboard type the bytes actually are, by magic number, or nil when /// they are none of the three the Java side can hand us. diff --git a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java index c8cfe878a4c..efaabe88f56 100644 --- a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java +++ b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java @@ -23,6 +23,11 @@ package com.codename1.impl.ios; import com.codename1.ui.Desktop; +import com.codename1.ui.ClipboardContent; +import com.codename1.ui.ClipboardDataProvider; +import com.codename1.ui.EncodedImage; +import com.codename1.ui.NativeDragAndDrop; +import com.codename1.ui.NativeDragOperation; import com.codename1.background.BackgroundFetch; import com.codename1.capture.VideoCaptureConstraints; import com.codename1.codescan.CodeScanner; @@ -9131,6 +9136,177 @@ public Object getPasteDataFromClipboard() { return super.getPasteDataFromClipboard(); } + + // ------------------------------------------------------------------------------------ + // Native drag and drop. + // + // UIKit owns the drag gesture: UIDragInteraction has its own recognizer and asks what is + // being dragged when it fires, so the framework cannot start a session on its own drag + // threshold the way the desktop port does. It stages the operation on the press instead, + // and CN1DragAndDrop.m calls nativeDragSessionStartedCallback below when UIKit decides a + // drag has begun. The payload is fetched at that moment rather than on the press, so a + // drag offering a file the application has not written yet does not write it every time + // the user merely touches the component. + // ------------------------------------------------------------------------------------ + + @Override + public boolean isNativeDragAndDropSupported() { + return nativeInstance.isNativeDragAndDropSupported(); + } + + @Override + public boolean isNativeDragOutsideApplicationSupported() { + return nativeInstance.isNativeDragOutsideAppSupported(); + } + + @Override + public boolean isNativeDragImageNeededOnPrepare() { + // UIKit asks for the lift preview at the instant its own recognizer fires, which is + // not a moment at which this port can render a component. + return true; + } + + @Override + public void prepareNativeDrag(NativeDragOperation op) { + ClipboardContent content = op.getContent(); + nativeInstance.prepareNativeDrag(join(content.getMimeTypes()), op.getAllowedActions(), + pngBytes(op.getDragImage()), op.getDragImageOffsetX(), op.getDragImageOffsetY()); + } + + @Override + public void cancelNativeDrag() { + nativeInstance.cancelNativeDrag(); + } + + /// Encodes a drag preview as PNG, the one image format the whole bridge speaks. + private static byte[] pngBytes(Image image) { + if (image == null) { + return null; + } + try { + return EncodedImage.createFromImage(image, false).getImageData(); + } catch (Throwable err) { + com.codename1.io.Log.e(err); + return null; + } + } + + private static String join(String[] values) { + if (values == null || values.length == 0) { + return null; + } + StringBuilder out = new StringBuilder(); + for (int iter = 0; iter < values.length; iter++) { + if (iter > 0) { + out.append('\n'); + } + out.append(values[iter]); + } + return out.toString(); + } + + private static String[] split(String value) { + if (value == null || value.length() == 0) { + return null; + } + java.util.List out = new java.util.ArrayList(); + int start = 0; + while (start <= value.length()) { + int next = value.indexOf('\n', start); + String entry = next < 0 ? value.substring(start) : value.substring(start, next); + if (entry.length() > 0) { + out.add(entry); + } + if (next < 0) { + break; + } + start = next + 1; + } + return out.isEmpty() ? null : out.toArray(new String[out.size()]); + } + + /// Describes a drag in progress from its MIME types alone. UIKit hands over no data until + /// the drop, and a drop target only needs the names in order to say whether it wants the + /// drag; the providers registered here answer null until the real content arrives. + private static ClipboardContent describe(String mimeTypes) { + ClipboardContent content = new ClipboardContent(); + String[] mimes = split(mimeTypes); + if (mimes == null) { + return content; + } + for (int iter = 0; iter < mimes.length; iter++) { + content.setDataProvider(mimes[iter], new ClipboardDataProvider() { + public Object getClipboardData(String mimeType) { + return null; + } + }); + } + return content; + } + + /// Invoked from CN1DragAndDrop.m as a drag moves over the surface. Returns the action a + /// drop would perform right now, or zero. + public static int nativeDragOverCallback(int x, int y, String mimeTypes, int allowedActions, + boolean entering) { + ClipboardContent content = describe(mimeTypes); + if (entering) { + return NativeDragAndDrop.dragEnter(0, x, y, content, allowedActions); + } + return NativeDragAndDrop.dragOver(0, x, y, content, allowedActions); + } + + /// Invoked from CN1DragAndDrop.m when a drag leaves the surface without dropping. + public static void nativeDragExitCallback() { + NativeDragAndDrop.dragExit(0); + } + + /// Invoked from CN1DragAndDrop.m with a fully loaded drop. Returns the action accepted, or + /// zero when nothing under the pointer took it. + public static int nativeDropCallback(int x, int y, String plain, String html, String rtf, + byte[] image, String fileUris, int action) { + ClipboardContent content = new ClipboardContent(); + if (plain != null) { + content.setData(ClipboardContent.MIME_TEXT, plain); + } + if (html != null) { + content.setData(ClipboardContent.MIME_HTML, html); + } + if (rtf != null) { + content.setData(ClipboardContent.MIME_RTF, rtf); + } + if (image != null && image.length > 0) { + content.setData(ClipboardContent.MIME_PNG, image); + } + content.setFiles(split(fileUris)); + return NativeDragAndDrop.drop(0, x, y, content, action); + } + + /// Invoked from CN1DragAndDrop.m when UIKit starts a drag session. Hands the payload down + /// -- which is where a promised representation is finally built -- and returns the actions + /// the operation allows, or zero when the framework has nothing staged and no drag should + /// begin. + public static int nativeDragSessionStartedCallback() { + NativeDragOperation op = NativeDragAndDrop.dragSessionStarted(); + if (op == null) { + return 0; + } + ClipboardContent content = op.getContent(); + byte[] image = content.getBytes(ClipboardContent.MIME_PNG); + if (image == null) { + image = content.getBytes(ClipboardContent.MIME_JPEG); + } + nativeInstance.setNativeDragPayload(content.getText(ClipboardContent.MIME_TEXT), + content.getText(ClipboardContent.MIME_HTML), + content.getText(ClipboardContent.MIME_RTF), + image, join(content.getFiles())); + return op.getAllowedActions(); + } + + /// Invoked from CN1DragAndDrop.m when a session this application started has ended. + public static void nativeDragCompletedCallback(int action) { + NativeDragAndDrop.dragCompleted(action); + } + @Override public void copyToClipboard(Object obj) { if(obj instanceof com.codename1.ui.ClipboardContent) { diff --git a/Ports/iOSPort/src/com/codename1/impl/ios/IOSNative.java b/Ports/iOSPort/src/com/codename1/impl/ios/IOSNative.java index bf685409486..8ac6afa9886 100644 --- a/Ports/iOSPort/src/com/codename1/impl/ios/IOSNative.java +++ b/Ports/iOSPort/src/com/codename1/impl/ios/IOSNative.java @@ -348,6 +348,54 @@ native void fillGradient(int kind, int stopCount, float[] positions, float[] pre native String getClipboardContent(String mimeType); native byte[] getClipboardImage(); native String getClipboardFileUris(); + + /// Native drag and drop. UIKit owns the drag gesture, so the framework stages what a press + /// has made draggable and the native side asks for the payload once UIDragInteraction + /// decides a drag has begun. See `Ports/iOSPort/nativeSources/CN1DragAndDrop.m`. + native boolean isNativeDragAndDropSupported(); + + /// True where a drag started in this application can be dropped in another one, which is + /// iPadOS and Mac Catalyst rather than a phone in full screen. + native boolean isNativeDragOutsideAppSupported(); + + /// Stages the representations a drag could offer -- named, not built -- along with what a + /// receiver may do with them and the image to show under the finger. + /// + /// #### Parameters + /// + /// - `mimeTypes`: newline separated MIME types + /// + /// - `allowedActions`: the `com.codename1.ui.NativeDragOperation` action bit set + /// + /// - `dragImagePng`: the preview as PNG bytes, or null for the platform default + /// + /// - `touchX`: the press position within the drag image + /// + /// - `touchY`: the press position within the drag image + native void prepareNativeDrag(String mimeTypes, int allowedActions, byte[] dragImagePng, + int touchX, int touchY); + + /// Delivers the payload for the session UIKit has just started, called from inside the + /// session-started callback so that a promised file is written only once a drag really + /// happens. + /// + /// #### Parameters + /// + /// - `plain`: the plain text representation, or null + /// + /// - `html`: the HTML representation, or null + /// + /// - `rtf`: the rich text representation, or null + /// + /// - `image`: image bytes, or null + /// + /// - `fileUris`: newline separated file paths or `file:` URIs, or null + native void setNativeDragPayload(String plain, String html, String rtf, byte[] image, + String fileUris); + + /// Drops whatever `#prepareNativeDrag(java.lang.String, int, byte[], int, int)` staged, + /// because the press turned out to be a tap. + native void cancelNativeDrag(); native void setPinchToZoomEnabled(long peer, boolean e); native void setNativeBrowserScrollingEnabled(long peer, boolean e); diff --git a/Samples/samples/NativeDragAndDropSample/NativeDragAndDropSample.java b/Samples/samples/NativeDragAndDropSample/NativeDragAndDropSample.java new file mode 100644 index 00000000000..30565ad3ee8 --- /dev/null +++ b/Samples/samples/NativeDragAndDropSample/NativeDragAndDropSample.java @@ -0,0 +1,224 @@ +/* + * Copyright (c) 2026, Codename One and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. Codename One designates this + * particular file as subject to the "Classpath" exception as provided + * by Oracle in the LICENSE file that accompanied this code. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Codename One through http://www.codenameone.com/ if you + * need additional information or have any questions. + */ + + +package com.codename1.samples; + +import static com.codename1.ui.CN.*; + +import com.codename1.io.FileSystemStorage; +import com.codename1.io.Log; +import com.codename1.ui.ClipboardContent; +import com.codename1.ui.ClipboardDataProvider; +import com.codename1.ui.Component; +import com.codename1.ui.Container; +import com.codename1.ui.Dialog; +import com.codename1.ui.Form; +import com.codename1.ui.Label; +import com.codename1.ui.NativeDragAndDrop; +import com.codename1.ui.NativeDragOperation; +import com.codename1.ui.NativeDropEvent; +import com.codename1.ui.Toolbar; +import com.codename1.ui.layouts.BorderLayout; +import com.codename1.ui.layouts.BoxLayout; +import com.codename1.ui.plaf.Border; +import com.codename1.ui.plaf.UIManager; +import com.codename1.ui.util.Resources; + +import java.io.OutputStream; + +/** + * Native operating system drag and drop: dragging content out of the application, and accepting + * a drag that came from somewhere else. + * + *

Drag the first card onto a text editor and the text lands there; drag the second onto the + * desktop or into a file manager and a real file appears -- written only at the moment of the + * drop, because the payload promises it rather than building it. Drag anything from another + * application onto the drop zone and it is described there.

+ */ +public class NativeDragAndDropSample { + + private Form current; + private Resources theme; + private Label status; + + public void init(Object context) { + updateNetworkThreadCount(2); + theme = UIManager.initFirstTheme("/theme"); + Toolbar.setGlobalToolbar(true); + Log.bindCrashProtection(true); + } + + public void start() { + if (current != null) { + current.show(); + return; + } + Form f = new Form("Native Drag and Drop", new BorderLayout()); + + status = new Label(NativeDragAndDrop.isSupported() + ? (NativeDragAndDrop.isDragOutsideApplicationSupported() + ? "Drags can leave this application" + : "Drags work inside this application only") + : "This platform has no native drag and drop"); + + Container body = new Container(BoxLayout.y()); + body.add(status); + body.add(textDragSource()); + body.add(fileDragSource()); + body.add(dropZone()); + + f.add(BorderLayout.CENTER, body); + f.show(); + } + + /** A card that drags plain text and HTML, so every receiver takes the best form it knows. */ + private Component textDragSource() { + Label card = new Label("Drag me into a text editor"); + card.getStyle().setBorder(Border.createLineBorder(2, 0x3366cc)); + card.getAllStyles().setPadding(8, 8, 8, 8); + + ClipboardContent content = new ClipboardContent() + .setData(ClipboardContent.MIME_TEXT, "Dragged out of Codename One") + .setData(ClipboardContent.MIME_HTML, + "Dragged out of Codename One"); + card.setNativeDragOperation(new NativeDragOperation(content) + .setAllowedActions(NativeDragOperation.ACTION_COPY) + .setLabel("Codename One text")); + return card; + } + + /** + * A card that drags a file which does not exist yet. + * + *

The file is registered as a provider rather than written up front, so a drag the user + * abandons costs nothing: the provider only runs if a receiver actually asks for the file + * list, which is what dropping on the desktop or in a file manager does.

+ */ + private Component fileDragSource() { + Label card = new Label("Drag me onto the desktop"); + card.getStyle().setBorder(Border.createLineBorder(2, 0x33aa55)); + card.getAllStyles().setPadding(8, 8, 8, 8); + + ClipboardContent content = new ClipboardContent() + .setData(ClipboardContent.MIME_TEXT, "codenameone-note.txt") + .setDataProvider(ClipboardContent.MIME_FILE, new ClipboardDataProvider() { + public Object getClipboardData(String mimeType) { + return writeNote(); + } + }); + + NativeDragOperation op = new NativeDragOperation(content) + .setAllowedActions(NativeDragOperation.ACTION_COPY) + .setLabel("codenameone-note.txt"); + op.addCompletionListener(e -> { + NativeDragOperation done = (NativeDragOperation) e.getSource(); + setStatus(done.getPerformedAction() == NativeDragOperation.ACTION_NONE + ? "The file drag was cancelled" + : "The file was dropped"); + }); + card.setNativeDragOperation(op); + return card; + } + + /** Writes the promised file and returns its path, or null when it could not be written. */ + private String writeNote() { + FileSystemStorage fs = FileSystemStorage.getInstance(); + String path = fs.getAppHomePath() + "codenameone-note.txt"; + try { + OutputStream out = fs.openOutputStream(path); + try { + out.write("Written by Codename One when you dropped it.\n".getBytes("UTF-8")); + } finally { + out.close(); + } + return path; + } catch (Exception err) { + Log.e(err); + return null; + } + } + + /** A zone that accepts anything dropped on it, from this application or from another one. */ + private Component dropZone() { + final Container zone = new Container(BoxLayout.y()) { + @Override + protected void nativeDragEnter(NativeDropEvent ev) { + getAllStyles().setBgColor(0xddeeff); + getAllStyles().setBgTransparency(255); + repaint(); + } + + @Override + protected void nativeDragExit(NativeDropEvent ev) { + getAllStyles().setBgTransparency(0); + repaint(); + } + + @Override + protected void nativeDrop(NativeDropEvent ev) { + getAllStyles().setBgTransparency(0); + repaint(); + } + }; + zone.getStyle().setBorder(Border.createDashedBorder(2, 0x888888)); + zone.getAllStyles().setPadding(16, 16, 16, 16); + zone.add(new Label("Drop anything here")); + zone.setNativeDropTarget(true); + zone.addNativeDropListener(e -> { + NativeDropEvent drop = (NativeDropEvent) e; + zone.removeAll(); + zone.add(new Label(drop.isLocal() ? "Dropped from this app" : "Dropped from elsewhere")); + String[] files = drop.getFiles(); + if (files != null) { + for (String file : files) { + zone.add(new Label(file)); + } + } else { + String text = drop.getText(); + zone.add(new Label(text == null ? "no text" : text)); + for (String mime : drop.getContent().getMimeTypes()) { + zone.add(new Label("offered: " + mime)); + } + } + zone.getComponentForm().revalidateWithAnimationSafety(); + }); + return zone; + } + + private void setStatus(String text) { + status.setText(text); + status.repaint(); + } + + public void stop() { + current = getCurrentForm(); + if (current instanceof Dialog) { + ((Dialog) current).dispose(); + current = getCurrentForm(); + } + } + + public void destroy() { + } +} diff --git a/docs/demos/common/src/main/java/com/codenameone/developerguide/advancedtopics/NativeDragAndDropDemo.java b/docs/demos/common/src/main/java/com/codenameone/developerguide/advancedtopics/NativeDragAndDropDemo.java new file mode 100644 index 00000000000..f631301e297 --- /dev/null +++ b/docs/demos/common/src/main/java/com/codenameone/developerguide/advancedtopics/NativeDragAndDropDemo.java @@ -0,0 +1,130 @@ +/* + * Copyright (c) 2026, Codename One and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. Codename One designates this + * particular file as subject to the "Classpath" exception as provided + * by Oracle in the LICENSE file that accompanied this code. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Codename One through http://www.codenameone.com/ if you + * need additional information or have any questions. + */ + + +package com.codenameone.developerguide.advancedtopics; + +import com.codename1.io.FileSystemStorage; +import com.codename1.ui.ClipboardContent; +import com.codename1.ui.ClipboardDataProvider; +import com.codename1.ui.Container; +import com.codename1.ui.Form; +import com.codename1.ui.Label; +import com.codename1.ui.NativeDragAndDrop; +import com.codename1.ui.NativeDragOperation; +import com.codename1.ui.NativeDropEvent; +import com.codename1.ui.layouts.BorderLayout; +import com.codename1.ui.layouts.BoxLayout; + +import java.io.OutputStream; + +public class NativeDragAndDropDemo { + + // tag::nativeDragSource[] + public void showDragSource() { + Form hi = new Form("Drag Out", BoxLayout.y()); + Label card = new Label("Drag me into another application"); + + ClipboardContent content = new ClipboardContent() + .setData(ClipboardContent.MIME_TEXT, "Dragged out of Codename One") + .setData(ClipboardContent.MIME_HTML, "Dragged out of Codename One"); + + card.setNativeDragOperation(new NativeDragOperation(content) + .setAllowedActions(NativeDragOperation.ACTION_COPY)); + + hi.add(card); + hi.show(); + } + // end::nativeDragSource[] + + // tag::nativeFileDrag[] + public void showFileDragSource() { + Form hi = new Form("Drag A File Out", BoxLayout.y()); + Label card = new Label("Drag me onto the desktop"); + + // The file is promised rather than written. The provider runs when a receiver actually + // asks for the file list, so a drag the user abandons costs nothing. + ClipboardContent content = new ClipboardContent() + .setData(ClipboardContent.MIME_TEXT, "note.txt") + .setDataProvider(ClipboardContent.MIME_FILE, new ClipboardDataProvider() { + public Object getClipboardData(String mimeType) { + return writeNote(); + } + }); + + NativeDragOperation op = new NativeDragOperation(content); + op.addCompletionListener(e -> + System.out.println("performed action: " + + ((NativeDragOperation) e.getSource()).getPerformedAction())); + card.setNativeDragOperation(op); + + hi.add(card); + hi.show(); + } + + private String writeNote() { + FileSystemStorage fs = FileSystemStorage.getInstance(); + String path = fs.getAppHomePath() + "note.txt"; + try { + OutputStream out = fs.openOutputStream(path); + try { + out.write("Written on drop\n".getBytes("UTF-8")); + } finally { + out.close(); + } + return path; + } catch (Exception err) { + return null; + } + } + // end::nativeFileDrag[] + + // tag::nativeDropTarget[] + public void showDropTarget() { + Form hi = new Form("Drop Here", new BorderLayout()); + Container zone = new Container(BoxLayout.y()); + zone.add(new Label("Drop files here")); + + zone.setNativeDropTarget(true); + zone.setAcceptedDropMimeTypes(ClipboardContent.MIME_FILE); + zone.setAcceptedDropActions(NativeDragOperation.ACTION_COPY); + zone.addNativeDropListener(e -> { + NativeDropEvent drop = (NativeDropEvent) e; + for (String path : drop.getFiles()) { + zone.add(new Label(path)); + } + zone.getComponentForm().revalidateWithAnimationSafety(); + }); + + hi.add(BorderLayout.CENTER, zone); + hi.show(); + } + // end::nativeDropTarget[] + + // tag::nativeDragSupport[] + public boolean canDragOut() { + return NativeDragAndDrop.isSupported() + && NativeDragAndDrop.isDragOutsideApplicationSupported(); + } + // end::nativeDragSupport[] +} diff --git a/docs/developer-guide/Advanced-Topics-Under-The-Hood.asciidoc b/docs/developer-guide/Advanced-Topics-Under-The-Hood.asciidoc index 373be8cb879..19febda4c1f 100644 --- a/docs/developer-guide/Advanced-Topics-Under-The-Hood.asciidoc +++ b/docs/developer-guide/Advanced-Topics-Under-The-Hood.asciidoc @@ -1257,7 +1257,9 @@ Unlike other platforms that tried to create overly generic catch all APIs Codena In Codename One components can be dragged and drop targets are always components. The logic of actually performing the operation indicated by the drop is the responsibility of the person implementing the drop. -NOTE: Some platforms for example: AWT allow dragging abstract concepts such as mime type elements. This allows dragging things like a text file into the app, but that use case isn't realistic in mobile +This lightweight dragging never leaves the application. To drag content into another +application, onto the desktop or into a file manager -- and to accept a drag coming the other +way -- see <<_native_operating_system_drag_drop>> below. The code below allows you to rearrange the items based on a sensible order. Notice it relies on the default `Container` drop behavior: @@ -1289,6 +1291,107 @@ In the drop target you can override the following methods: * `drop` - the logic for dropping/moving the component must be implemented here! +==== Native operating system drag & drop + +The section above moves a component around inside one form. Native dragging hands the gesture to +the operating system instead, so a drag can end in another application, on the desktop or in a +file manager, and a drag started anywhere on the machine can end on one of your components. + +The payload is a `ClipboardContent`, the same object a copy publishes. That's the whole idea: a +drag is a copy that the user aims with the pointer, so whatever the application can already put +on the clipboard it can already drag out, and whatever it can paste it can already accept as a +drop. Offering several representations lets one drag land correctly in unrelated applications -- +a text editor takes the `text/html` representation, a plain text field takes `text/plain`, and +the desktop takes the file list. + +To drag something out, give the component an operation: + +[source,java] +---- +include::../demos/common/src/main/java/com/codenameone/developerguide/advancedtopics/NativeDragAndDropDemo.java[tag=nativeDragSource,indent=0] +---- + +Dragging a file works the same way, except that the file often doesn't exist yet and the user +may drop it nowhere at all. Register it as a provider and it's written at the moment a receiver +asks for it, and never otherwise: + +[source,java] +---- +include::../demos/common/src/main/java/com/codenameone/developerguide/advancedtopics/NativeDragAndDropDemo.java[tag=nativeFileDrag,indent=0] +---- + +`NativeDragOperation.ACTION_MOVE` means the receiver takes ownership and the source deletes its +copy. The source only learns whether that happened once the operating system has finished, which +is why the outcome arrives through the completion listener rather than from the call that started +the drag. + +Receiving a drop is the mirror image. The deepest component under the pointer that's a native +drop target and accepts the content wins, so a target nested inside another takes precedence and +a target that refuses a particular payload lets an ancestor have it: + +[source,java] +---- +include::../demos/common/src/main/java/com/codenameone/developerguide/advancedtopics/NativeDragAndDropDemo.java[tag=nativeDropTarget,indent=0] +---- + +`NativeDropEvent` also reports whether the drag started inside this application, which is how a +container that reorders its own items tells that case from an import. + +===== Where it works + +Native dragging needs the platform to have it, and platforms differ on whether a drag may leave +the application at all -- a desktop can drop onto any other window, a tablet can drop into an +application beside it, and a phone in full screen has nowhere for a drag to go even though drags +within the application still work. Check both before offering the affordance: + +[source,java] +---- +include::../demos/common/src/main/java/com/codenameone/developerguide/advancedtopics/NativeDragAndDropDemo.java[tag=nativeDragSupport,indent=0] +---- + +Where the platform has none of this, the calls are harmless no-ops: the component simply can't be +dragged through the operating system, and the lightweight dragging above is unaffected. + +[cols="2,1,1,4"] +|=== +|Platform |Drags |Leaves the app |Notes + +|Desktop (the simulator and "run as desktop app") +|yes |yes +|Drops onto other windows, the desktop and file managers. Text, HTML, images, arbitrary +binary payloads and multiple files. + +|Android +|yes |Nougat and later +|A drag crosses applications through `DRAG_FLAG_GLOBAL`, which arrived in Nougat. Files +travel as content URIs, the same way the clipboard carries them. + +|iPadOS and Mac Catalyst +|yes |yes +|Drops into another application beside this one, and into Files or the Finder. + +|iPhone +|yes |no +|The session works, but a phone in full screen has no second application to drop into. + +|Everything else +|no |no +|`isSupported()` answers false and the lightweight dragging above is unaffected. +|=== + +===== Threading + +Drops arrive on the operating system's own drag thread rather than the event dispatch thread. The +framework resolves which component a drag is over on that thread, from the accepted MIME types +and actions alone, and dispatches the callbacks -- `nativeDragEnter`, `nativeDragOver`, +`nativeDragExit` and `nativeDrop` -- on the event dispatch thread. A filter set through +`setAcceptedDropMimeTypes` therefore takes effect from the first drag event, while a decision +made inside a callback reaches the cursor one event later. + +The one method that runs off the event dispatch thread is `canAcceptNativeDrop`, which exists for +a target whose answer depends on more than the MIME type. Read the content and the component's +own configuration there; leave the user interface alone. + === Android Lollipop ActionBar customization diff --git a/maven/core-unittests/src/test/java/com/codename1/testing/TestCodenameOneImplementation.java b/maven/core-unittests/src/test/java/com/codename1/testing/TestCodenameOneImplementation.java index fb6c878ed4b..6c91331057e 100644 --- a/maven/core-unittests/src/test/java/com/codename1/testing/TestCodenameOneImplementation.java +++ b/maven/core-unittests/src/test/java/com/codename1/testing/TestCodenameOneImplementation.java @@ -115,6 +115,15 @@ public class TestCodenameOneImplementation extends CodenameOneImplementation { private Message lastSentMessage; private int refreshContactsCount; + /// Native drag and drop, faked so the framework's outbound path can be exercised without a + /// real operating system drag: the last operation prepared and the last one started are + /// recorded, and startNativeDrag reports success only while nativeDragAndDropSupported is on. + private boolean nativeDragAndDropSupported; + private com.codename1.ui.NativeDragOperation preparedNativeDrag; + private com.codename1.ui.NativeDragOperation startedNativeDrag; + private int cancelledNativeDrags; + private boolean nativeDragStartRefused; + private final TestFont defaultFont = new TestFont(8, 16); private int displayWidth = 1080; private int displayHeight = 1920; @@ -5474,4 +5483,67 @@ public void windowPointerDraggedForTest(int windowId, int x, int y) { public void windowPointerReleasedForTest(int windowId, int x, int y) { windowPointerReleased(windowId, x, y); } + + // ------------------------------------------------------------------------------------ + // Native drag and drop test hooks + // ------------------------------------------------------------------------------------ + + /// Turns the fake native drag and drop on, which is what makes + /// `com.codename1.ui.NativeDragAndDrop#isSupported()` true for a test. + public void setNativeDragAndDropSupported(boolean supported) { + this.nativeDragAndDropSupported = supported; + } + + @Override + public boolean isNativeDragAndDropSupported() { + return nativeDragAndDropSupported; + } + + @Override + public void prepareNativeDrag(com.codename1.ui.NativeDragOperation op) { + preparedNativeDrag = op; + } + + @Override + public boolean startNativeDrag(com.codename1.ui.NativeDragOperation op) { + if (!nativeDragAndDropSupported || nativeDragStartRefused) { + return false; + } + startedNativeDrag = op; + return true; + } + + /// Makes startNativeDrag refuse, which is how a port whose operating system owns the drag + /// gesture behaves: it starts no session of its own and announces the platform's later. + public void setNativeDragStartRefused(boolean refused) { + this.nativeDragStartRefused = refused; + } + + @Override + public void cancelNativeDrag() { + cancelledNativeDrags++; + preparedNativeDrag = null; + } + + /// The operation the last press staged, or null. + public com.codename1.ui.NativeDragOperation getPreparedNativeDrag() { + return preparedNativeDrag; + } + + /// The operation the last drag actually started, or null. + public com.codename1.ui.NativeDragOperation getStartedNativeDrag() { + return startedNativeDrag; + } + + /// How many prepared operations were dropped because the press turned out to be a click. + public int getCancelledNativeDrags() { + return cancelledNativeDrags; + } + + /// Forgets everything recorded, so one test does not see another's drag. + public void resetNativeDragState() { + preparedNativeDrag = null; + startedNativeDrag = null; + cancelledNativeDrags = 0; + } } diff --git a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java new file mode 100644 index 00000000000..9fea5fbd932 --- /dev/null +++ b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java @@ -0,0 +1,487 @@ +/* + * Copyright (c) 2026, Codename One and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. Codename One designates this + * particular file as subject to the "Classpath" exception as provided + * by Oracle in the LICENSE file that accompanied this code. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Codename One through http://www.codenameone.com/ if you + * need additional information or have any questions. + */ + + +package com.codename1.ui; + +import com.codename1.junit.FormTest; +import com.codename1.junit.UITestBase; +import com.codename1.ui.events.ActionEvent; +import com.codename1.ui.layouts.BorderLayout; +import org.junit.jupiter.api.Test; + +import java.util.ArrayList; +import java.util.List; + +import static org.junit.jupiter.api.Assertions.*; + +/** + * Native (operating system) drag and drop: the payload container, the target resolution the + * ports rely on, and the gesture that hands a press to the platform. + * + *

The platform half is faked through {@code TestCodenameOneImplementation}, which records the + * operation the framework prepared and started instead of talking to a window system. What is + * under test here is everything above the port: which component a drag resolves to, what it + * answers the operating system, and which callbacks fire.

+ */ +class NativeDragAndDropTest extends UITestBase { + + /** A component that records every native drag callback it receives. */ + private static final class DropRecorder extends Container { + final List events = new ArrayList(); + ClipboardContent dropped; + int rejectAction = -1; + + @Override + protected void nativeDragEnter(NativeDropEvent ev) { + events.add("enter"); + if (rejectAction >= 0) { + ev.accept(rejectAction); + } + } + + @Override + protected void nativeDragOver(NativeDropEvent ev) { + events.add("over"); + if (rejectAction >= 0) { + ev.accept(rejectAction); + } + } + + @Override + protected void nativeDragExit(NativeDropEvent ev) { + events.add("exit"); + } + + @Override + protected void nativeDrop(NativeDropEvent ev) { + events.add("drop"); + dropped = ev.getContent(); + } + } + + private DropRecorder addTarget(Form form) { + DropRecorder target = new DropRecorder(); + target.setNativeDropTarget(true); + form.setLayout(new BorderLayout()); + form.add(BorderLayout.CENTER, target); + form.revalidate(); + return target; + } + + private static ClipboardContent textContent(String text) { + return new ClipboardContent().setData(ClipboardContent.MIME_TEXT, text); + } + + // ------------------------------------------------------------------------------------ + // ClipboardContent as a drag payload + // ------------------------------------------------------------------------------------ + + @Test + void lazyRepresentationIsNotBuiltUntilItIsRead() { + final int[] calls = {0}; + ClipboardContent content = new ClipboardContent() + .setData(ClipboardContent.MIME_TEXT, "report") + .setDataProvider(ClipboardContent.MIME_FILE, new ClipboardDataProvider() { + public Object getClipboardData(String mimeType) { + calls[0]++; + return "/tmp/report.pdf"; + } + }); + + assertTrue(content.hasMimeType(ClipboardContent.MIME_FILE), + "a promised representation is advertised before it is built"); + assertEquals(0, calls[0], "advertising a representation must not build it"); + + assertArrayEquals(new String[]{"/tmp/report.pdf"}, content.getFiles()); + assertEquals(1, calls[0], "reading it builds it"); + assertArrayEquals(new String[]{"/tmp/report.pdf"}, content.getFiles()); + assertEquals(1, calls[0], "reading it again reuses the value rather than writing the file twice"); + } + + @Test + void fileListReadsBackWhicheverWayItWasStored() { + ClipboardContent one = new ClipboardContent().setFiles(new String[]{"/tmp/a.txt"}); + assertEquals("/tmp/a.txt", one.getData(ClipboardContent.MIME_FILE), + "a single file is stored as a plain string, which is what the ports expect"); + assertArrayEquals(new String[]{"/tmp/a.txt"}, one.getFiles()); + + ClipboardContent many = new ClipboardContent().setFiles(new String[]{"/tmp/a.txt", "/tmp/b.txt"}); + assertArrayEquals(new String[]{"/tmp/a.txt", "/tmp/b.txt"}, many.getFiles()); + + assertNull(new ClipboardContent().getFiles(), "no files means null rather than an empty array"); + assertNull(new ClipboardContent().setFiles(null).getFiles()); + } + + // On the event dispatch thread, because EventDispatcher defers a firing made from any other + // thread and the listener would then not have run by the time the assertion below reads it. + @FormTest + void anOperationDefaultsToCopyAndReportsNothingUntilItCompletes() { + NativeDragOperation op = new NativeDragOperation("hello"); + assertEquals(NativeDragOperation.ACTION_COPY, op.getAllowedActions()); + assertEquals("hello", op.getContent().getText(ClipboardContent.MIME_TEXT)); + assertEquals(NativeDragOperation.ACTION_NONE, op.getPerformedAction()); + + final int[] completed = {-1}; + op.addCompletionListener(e -> completed[0] = ((NativeDragOperation) e.getSource()).getPerformedAction()); + op.fireCompleted(NativeDragOperation.ACTION_MOVE); + assertEquals(NativeDragOperation.ACTION_MOVE, op.getPerformedAction()); + assertEquals(NativeDragOperation.ACTION_MOVE, completed[0], + "a source that offered a move learns here, and only here, that it must delete its copy"); + } + + // ------------------------------------------------------------------------------------ + // Resolving the target and answering the operating system + // ------------------------------------------------------------------------------------ + + @FormTest + void aDragOverATargetIsAcceptedAndDeliversEnterThenOver() { + Form form = Display.getInstance().getCurrent(); + DropRecorder target = addTarget(form); + + int x = target.getAbsoluteX() + 5; + int y = target.getAbsoluteY() + 5; + int action = NativeDragAndDrop.dragEnter(0, x, y, textContent("hi"), + NativeDragOperation.ACTION_COPY | NativeDragOperation.ACTION_MOVE); + assertEquals(NativeDragOperation.ACTION_COPY, action, + "a target that expresses no preference copies, which cannot destroy the source's data"); + + NativeDragAndDrop.dragOver(0, x + 1, y + 1, textContent("hi"), NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + assertEquals("enter", target.events.get(0)); + assertTrue(target.events.contains("over")); + + NativeDragAndDrop.dragExit(0); + flushSerialCalls(); + assertEquals("exit", target.events.get(target.events.size() - 1)); + } + + @FormTest + void aDragOverNothingIsRejected() { + Form form = Display.getInstance().getCurrent(); + addTarget(form); + form.setNativeDropTarget(false); + + // The title area is outside the target; nothing there accepts drops. + int action = NativeDragAndDrop.dragEnter(0, 1, 1, textContent("hi"), NativeDragOperation.ACTION_COPY); + assertEquals(NativeDragOperation.ACTION_NONE, action); + NativeDragAndDrop.dragExit(0); + flushSerialCalls(); + } + + @FormTest + void aMimeFilterRefusesTheDragFromTheVeryFirstEvent() { + Form form = Display.getInstance().getCurrent(); + DropRecorder target = addTarget(form); + target.setAcceptedDropMimeTypes(ClipboardContent.MIME_FILE); + + int x = target.getAbsoluteX() + 5; + int y = target.getAbsoluteY() + 5; + assertEquals(NativeDragOperation.ACTION_NONE, + NativeDragAndDrop.dragEnter(0, x, y, textContent("hi"), NativeDragOperation.ACTION_COPY), + "text is not a file, so the target never sees the drag at all"); + flushSerialCalls(); + assertTrue(target.events.isEmpty()); + + ClipboardContent files = new ClipboardContent().setFiles(new String[]{"/tmp/a.txt"}); + assertEquals(NativeDragOperation.ACTION_COPY, + NativeDragAndDrop.dragEnter(0, x, y, files, NativeDragOperation.ACTION_COPY)); + NativeDragAndDrop.dragExit(0); + flushSerialCalls(); + } + + @FormTest + void anActionTheTargetRefusesIsNotOffered() { + Form form = Display.getInstance().getCurrent(); + DropRecorder target = addTarget(form); + target.setAcceptedDropActions(NativeDragOperation.ACTION_MOVE); + + int x = target.getAbsoluteX() + 5; + int y = target.getAbsoluteY() + 5; + assertEquals(NativeDragOperation.ACTION_NONE, + NativeDragAndDrop.dragEnter(0, x, y, textContent("hi"), NativeDragOperation.ACTION_COPY), + "a copy-only source and a move-only target have nothing in common"); + NativeDragAndDrop.dragExit(0); + + assertEquals(NativeDragOperation.ACTION_MOVE, + NativeDragAndDrop.dragEnter(0, x, y, textContent("hi"), + NativeDragOperation.ACTION_COPY | NativeDragOperation.ACTION_MOVE)); + NativeDragAndDrop.dragExit(0); + flushSerialCalls(); + } + + @FormTest + void theDeepestTargetWins() { + Form form = Display.getInstance().getCurrent(); + DropRecorder outer = addTarget(form); + DropRecorder inner = new DropRecorder(); + inner.setNativeDropTarget(true); + outer.setLayout(new BorderLayout()); + outer.add(BorderLayout.CENTER, inner); + form.revalidate(); + + NativeDragAndDrop.drop(0, inner.getAbsoluteX() + 2, inner.getAbsoluteY() + 2, + textContent("hi"), NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + assertTrue(inner.events.contains("drop")); + assertFalse(outer.events.contains("drop")); + } + + @FormTest + void aTargetThatRefusesThisPayloadLetsAnAncestorHaveIt() { + Form form = Display.getInstance().getCurrent(); + DropRecorder outer = addTarget(form); + DropRecorder inner = new DropRecorder(); + inner.setNativeDropTarget(true); + inner.setAcceptedDropMimeTypes(ClipboardContent.MIME_FILE); + outer.setLayout(new BorderLayout()); + outer.add(BorderLayout.CENTER, inner); + form.revalidate(); + + NativeDragAndDrop.drop(0, inner.getAbsoluteX() + 2, inner.getAbsoluteY() + 2, + textContent("hi"), NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + assertFalse(inner.events.contains("drop")); + assertTrue(outer.events.contains("drop")); + } + + @FormTest + void dropDeliversTheContentAndNotifiesTheListener() { + Form form = Display.getInstance().getCurrent(); + DropRecorder target = addTarget(form); + final NativeDropEvent[] seen = new NativeDropEvent[1]; + target.addNativeDropListener(e -> seen[0] = (NativeDropEvent) e); + + ClipboardContent content = new ClipboardContent() + .setData(ClipboardContent.MIME_TEXT, "two files") + .setFiles(new String[]{"/tmp/a.txt", "/tmp/b.txt"}); + int accepted = NativeDragAndDrop.drop(0, target.getAbsoluteX() + 5, target.getAbsoluteY() + 5, + content, NativeDragOperation.ACTION_COPY); + assertEquals(NativeDragOperation.ACTION_COPY, accepted); + flushSerialCalls(); + + assertNotNull(target.dropped); + assertArrayEquals(new String[]{"/tmp/a.txt", "/tmp/b.txt"}, target.dropped.getFiles(), + "a drop of several files arrives as several files"); + assertNotNull(seen[0]); + assertEquals(ActionEvent.Type.NativeDrop, seen[0].getEventType()); + assertEquals("two files", seen[0].getText()); + assertFalse(seen[0].isLocal(), "a drag this application did not start is not local"); + } + + @FormTest + void dropOnNothingReportsFailureSoThePortCanTellTheSource() { + Form form = Display.getInstance().getCurrent(); + addTarget(form); + assertEquals(NativeDragOperation.ACTION_NONE, + NativeDragAndDrop.drop(0, 1, 1, textContent("hi"), NativeDragOperation.ACTION_COPY)); + flushSerialCalls(); + } + + // ------------------------------------------------------------------------------------ + // The gesture: a press on a drag source becomes an operating system drag + // ------------------------------------------------------------------------------------ + + @FormTest + void aDragOnANativeDragSourceIsHandedToThePlatform() { + implementation.resetNativeDragState(); + implementation.setNativeDragAndDropSupported(true); + try { + Form form = Display.getInstance().getCurrent(); + Container source = new Container(); + source.setNativeDragOperation(new NativeDragOperation("dragged out")); + form.setLayout(new BorderLayout()); + form.add(BorderLayout.CENTER, source); + form.revalidate(); + + int x = source.getAbsoluteX() + 10; + int y = source.getAbsoluteY() + 10; + form.pointerPressed(x, y); + assertNotNull(implementation.getPreparedNativeDrag(), + "the press stages the payload so a platform that owns the gesture can ask for it"); + assertNull(implementation.getStartedNativeDrag(), "a press alone is not a drag"); + + form.pointerDragged(x + 200, y + 200); + assertNotNull(implementation.getStartedNativeDrag(), "moving far enough starts the session"); + assertSame(source, implementation.getStartedNativeDrag().getSource()); + assertSame(implementation.getStartedNativeDrag(), NativeDragAndDrop.getActiveDrag()); + + NativeDragAndDrop.dragCompleted(NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + assertNull(NativeDragAndDrop.getActiveDrag()); + } finally { + implementation.setNativeDragAndDropSupported(false); + implementation.resetNativeDragState(); + } + } + + @FormTest + void aDragSourceInsideADraggableContainerIsStillStaged() { + implementation.resetNativeDragState(); + implementation.setNativeDragAndDropSupported(true); + try { + Form form = Display.getInstance().getCurrent(); + // The form primes drag and drop on the pressed component and then again on its + // nearest draggable ancestor. The drag source sits between the two, so the second + // pass cannot find it -- and must not throw away what the first pass staged. + Container draggableOuter = new Container(new BorderLayout()); + draggableOuter.setDraggable(true); + Container source = new Container(); + source.setNativeDragOperation(new NativeDragOperation("from the middle")); + draggableOuter.add(BorderLayout.CENTER, source); + form.setLayout(new BorderLayout()); + form.add(BorderLayout.CENTER, draggableOuter); + form.revalidate(); + + int x = source.getAbsoluteX() + 10; + int y = source.getAbsoluteY() + 10; + form.pointerPressed(x, y); + assertNotNull(implementation.getPreparedNativeDrag()); + + form.pointerDragged(x + 200, y + 200); + assertNotNull(implementation.getStartedNativeDrag()); + assertSame(source, implementation.getStartedNativeDrag().getSource()); + + NativeDragAndDrop.dragCompleted(NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + } finally { + implementation.setNativeDragAndDropSupported(false); + implementation.resetNativeDragState(); + } + } + + @FormTest + void aClickOnANativeDragSourceDragsNothing() { + implementation.resetNativeDragState(); + implementation.setNativeDragAndDropSupported(true); + try { + Form form = Display.getInstance().getCurrent(); + Container source = new Container(); + source.setNativeDragOperation(new NativeDragOperation("dragged out")); + form.setLayout(new BorderLayout()); + form.add(BorderLayout.CENTER, source); + form.revalidate(); + + int x = source.getAbsoluteX() + 10; + int y = source.getAbsoluteY() + 10; + form.pointerPressed(x, y); + form.pointerReleased(x, y); + assertNull(implementation.getStartedNativeDrag()); + assertEquals(1, implementation.getCancelledNativeDrags(), + "the staged payload is dropped, so a later gesture elsewhere cannot start this drag"); + } finally { + implementation.setNativeDragAndDropSupported(false); + implementation.resetNativeDragState(); + } + } + + @FormTest + void aPlatformThatOwnsTheGestureKeepsTheStagedOperationUntilItStarts() { + implementation.resetNativeDragState(); + implementation.setNativeDragAndDropSupported(true); + // startNativeDrag refuses, which is what a port whose operating system owns the drag + // gesture looks like: iOS starts the session from its own long press and announces it + // afterwards through dragSessionStarted(). + implementation.setNativeDragStartRefused(true); + try { + Form form = Display.getInstance().getCurrent(); + Container source = new Container(); + NativeDragOperation op = new NativeDragOperation("dragged out"); + source.setNativeDragOperation(op); + form.setLayout(new BorderLayout()); + form.add(BorderLayout.CENTER, source); + form.revalidate(); + + int x = source.getAbsoluteX() + 10; + int y = source.getAbsoluteY() + 10; + form.pointerPressed(x, y); + form.pointerDragged(x + 200, y + 200); + assertNull(NativeDragAndDrop.getActiveDrag(), + "the port refused, so no session is running yet"); + // A second drag packet must not offer the same gesture again. + form.pointerDragged(x + 220, y + 220); + + assertSame(op, NativeDragAndDrop.dragSessionStarted(), + "the staged operation is still there for the platform's own recognizer"); + assertSame(op, NativeDragAndDrop.getActiveDrag()); + assertNull(NativeDragAndDrop.dragSessionStarted(), + "and it is only handed over once"); + + NativeDragAndDrop.dragCompleted(NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + assertEquals(NativeDragOperation.ACTION_COPY, op.getPerformedAction()); + } finally { + implementation.setNativeDragStartRefused(false); + implementation.setNativeDragAndDropSupported(false); + implementation.resetNativeDragState(); + NativeDragAndDrop.dragCompleted(NativeDragOperation.ACTION_NONE); + } + } + + @FormTest + void aReleaseAfterARefusedStartDoesNotLeaveTheDragArmed() { + implementation.resetNativeDragState(); + implementation.setNativeDragAndDropSupported(true); + implementation.setNativeDragStartRefused(true); + try { + Form form = Display.getInstance().getCurrent(); + Container source = new Container(); + source.setNativeDragOperation(new NativeDragOperation("dragged out")); + form.setLayout(new BorderLayout()); + form.add(BorderLayout.CENTER, source); + form.revalidate(); + + int x = source.getAbsoluteX() + 10; + int y = source.getAbsoluteY() + 10; + form.pointerPressed(x, y); + form.pointerDragged(x + 200, y + 200); + form.pointerReleased(x + 200, y + 200); + assertNull(NativeDragAndDrop.dragSessionStarted(), + "a gesture that ended cannot be turned into a drag by a later recognizer"); + } finally { + implementation.setNativeDragStartRefused(false); + implementation.setNativeDragAndDropSupported(false); + implementation.resetNativeDragState(); + } + } + + @FormTest + void withoutPlatformSupportTheGestureIsLeftAlone() { + implementation.resetNativeDragState(); + implementation.setNativeDragAndDropSupported(false); + Form form = Display.getInstance().getCurrent(); + Container source = new Container(); + source.setNativeDragOperation(new NativeDragOperation("dragged out")); + form.setLayout(new BorderLayout()); + form.add(BorderLayout.CENTER, source); + form.revalidate(); + + int x = source.getAbsoluteX() + 10; + int y = source.getAbsoluteY() + 10; + form.pointerPressed(x, y); + form.pointerDragged(x + 200, y + 200); + assertNull(implementation.getPreparedNativeDrag()); + assertNull(implementation.getStartedNativeDrag()); + form.pointerReleased(x + 200, y + 200); + } +} diff --git a/maven/javase/src/test/java/com/codename1/impl/javase/JavaSENativeDragAndDropTest.java b/maven/javase/src/test/java/com/codename1/impl/javase/JavaSENativeDragAndDropTest.java new file mode 100644 index 00000000000..0a9290dc68f --- /dev/null +++ b/maven/javase/src/test/java/com/codename1/impl/javase/JavaSENativeDragAndDropTest.java @@ -0,0 +1,256 @@ +/* + * Copyright (c) 2026, Codename One and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. Codename One designates this + * particular file as subject to the "Classpath" exception as provided + * by Oracle in the LICENSE file that accompanied this code. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Codename One through http://www.codenameone.com/ if you + * need additional information or have any questions. + */ + + +package com.codename1.impl.javase; + +import com.codename1.ui.ClipboardContent; +import com.codename1.ui.ClipboardDataProvider; +import com.codename1.ui.NativeDragOperation; +import org.junit.jupiter.api.Test; + +import java.awt.datatransfer.DataFlavor; +import java.awt.datatransfer.Transferable; +import java.awt.datatransfer.UnsupportedFlavorException; +import java.io.ByteArrayInputStream; +import java.io.File; +import java.io.IOException; +import java.io.InputStream; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +import static org.junit.jupiter.api.Assertions.*; + +/** + * The two conversions that decide whether a desktop drag lands correctly: the + * {@code ClipboardContent} the application offers becomes an AWT transferable other applications + * understand, and the transferable another application drops becomes a {@code ClipboardContent}. + * + *

These run headless -- they never open a window or start a real drag -- because everything + * platform specific about the drag is AWT's, while everything that can be wrong is in the + * mapping.

+ */ +class JavaSENativeDragAndDropTest { + + private static DataFlavor flavorFor(Transferable t, String mime) { + for (DataFlavor f : t.getTransferDataFlavors()) { + if (f.isMimeTypeEqual(mime)) { + return f; + } + } + return null; + } + + /** A transferable that serves fixed values, standing in for another application's drag. */ + private static final class FakeTransferable implements Transferable { + private final List flavors = new ArrayList(); + private final List values = new ArrayList(); + int reads; + + FakeTransferable add(DataFlavor flavor, Object value) { + flavors.add(flavor); + values.add(value); + return this; + } + + public DataFlavor[] getTransferDataFlavors() { + return flavors.toArray(new DataFlavor[flavors.size()]); + } + + public boolean isDataFlavorSupported(DataFlavor flavor) { + return flavors.contains(flavor); + } + + public Object getTransferData(DataFlavor flavor) throws UnsupportedFlavorException, IOException { + int index = flavors.indexOf(flavor); + if (index < 0) { + throw new UnsupportedFlavorException(flavor); + } + reads++; + return values.get(index); + } + } + + // ------------------------------------------------------------------------------------ + // Dragging out + // ------------------------------------------------------------------------------------ + + @Test + void richTextIsOfferedInEveryFormatItWasGiven() throws Exception { + ClipboardContent content = new ClipboardContent() + .setData(ClipboardContent.MIME_TEXT, "hello") + .setData(ClipboardContent.MIME_HTML, "hello"); + Transferable t = new JavaSEPort.RichTransferable(content); + + assertTrue(t.isDataFlavorSupported(DataFlavor.stringFlavor)); + assertEquals("hello", t.getTransferData(DataFlavor.stringFlavor)); + + DataFlavor html = flavorFor(t, ClipboardContent.MIME_HTML); + assertNotNull(html, "a rich text editor asks for text/html and has to find it"); + assertEquals("hello", t.getTransferData(html)); + } + + @Test + void filesAreOfferedBothAsAFileListAndAsAUriList() throws Exception { + File a = File.createTempFile("cn1-dnd-a", ".txt"); + File b = File.createTempFile("cn1-dnd-b", ".txt"); + a.deleteOnExit(); + b.deleteOnExit(); + ClipboardContent content = new ClipboardContent() + .setData(ClipboardContent.MIME_TEXT, "two files") + .setFiles(new String[]{a.getAbsolutePath(), b.getAbsolutePath()}); + Transferable t = new JavaSEPort.RichTransferable(content); + + assertTrue(t.isDataFlavorSupported(DataFlavor.javaFileListFlavor), + "a drop on the desktop or in a file manager reads the file list flavor"); + List files = (List) t.getTransferData(DataFlavor.javaFileListFlavor); + assertEquals(2, files.size(), "a drag of several files stays a drag of several files"); + assertEquals(a.getAbsolutePath(), ((File) files.get(0)).getAbsolutePath()); + + DataFlavor uriList = flavorFor(t, ClipboardContent.MIME_URI_LIST); + assertNotNull(uriList, "GTK targets ask for text/uri-list and nothing else"); + String uris = (String) t.getTransferData(uriList); + assertTrue(uris.contains(a.toURI().toString())); + assertTrue(uris.contains(b.toURI().toString())); + } + + @Test + void aPromisedFileIsNotBuiltUntilTheDropReadsIt() throws Exception { + final int[] built = {0}; + final File promised = File.createTempFile("cn1-dnd-promise", ".txt"); + promised.deleteOnExit(); + ClipboardContent content = new ClipboardContent() + .setData(ClipboardContent.MIME_TEXT, "report") + .setDataProvider(ClipboardContent.MIME_FILE, new ClipboardDataProvider() { + public Object getClipboardData(String mimeType) { + built[0]++; + return promised.getAbsolutePath(); + } + }); + + Transferable t = new JavaSEPort.RichTransferable(content); + assertTrue(t.isDataFlavorSupported(DataFlavor.javaFileListFlavor), + "the file flavor is advertised from the MIME type alone"); + assertEquals(0, built[0], + "starting the drag must not write the file -- the user may drop it nowhere"); + + List files = (List) t.getTransferData(DataFlavor.javaFileListFlavor); + assertEquals(1, files.size()); + assertEquals(1, built[0], "the drop is what builds it"); + } + + @Test + void binaryContentIsOfferedAsAStream() throws Exception { + byte[] pdf = new byte[]{'%', 'P', 'D', 'F'}; + ClipboardContent content = new ClipboardContent() + .setData(ClipboardContent.MIME_TEXT, "doc") + .setData("application/pdf", pdf); + Transferable t = new JavaSEPort.RichTransferable(content); + + DataFlavor flavor = flavorFor(t, "application/pdf"); + assertNotNull(flavor, "an arbitrary binary payload has to reach other applications somehow"); + Object value = t.getTransferData(flavor); + assertTrue(value instanceof InputStream); + byte[] read = new byte[4]; + ((InputStream) value).read(read); + assertArrayEquals(pdf, read); + } + + @Test + void anUnofferedFlavorIsRefusedRatherThanAnsweredWithNull() { + ClipboardContent content = new ClipboardContent().setData(ClipboardContent.MIME_TEXT, "hi"); + final Transferable t = new JavaSEPort.RichTransferable(content); + assertThrows(UnsupportedFlavorException.class, + () -> t.getTransferData(DataFlavor.javaFileListFlavor)); + } + + // ------------------------------------------------------------------------------------ + // Receiving a drop + // ------------------------------------------------------------------------------------ + + @Test + void describingADragInProgressReadsNoData() { + FakeTransferable t = new FakeTransferable() + .add(DataFlavor.stringFlavor, "hello") + .add(DataFlavor.javaFileListFlavor, Arrays.asList(new File("/tmp/a.txt"))); + + ClipboardContent content = JavaSENativeDragAndDrop.contentFor(t, t.getTransferDataFlavors(), false); + assertTrue(content.hasMimeType(ClipboardContent.MIME_TEXT)); + assertTrue(content.hasMimeType(ClipboardContent.MIME_FILE)); + assertEquals(0, t.reads, + "a drag merely passing over the window must not pull data across; on several " + + "platforms the data does not exist until the drop"); + } + + @Test + void aDroppedFileListBecomesFilePaths() { + FakeTransferable t = new FakeTransferable() + .add(DataFlavor.javaFileListFlavor, + Arrays.asList(new File("/tmp/a.txt"), new File("/tmp/b.txt"))); + + ClipboardContent content = JavaSENativeDragAndDrop.contentFor(t, t.getTransferDataFlavors(), true); + assertArrayEquals(new String[]{"/tmp/a.txt", "/tmp/b.txt"}, content.getFiles()); + } + + @Test + void aDroppedUriListAlsoBecomesFilePaths() throws Exception { + DataFlavor uriList = new DataFlavor(ClipboardContent.MIME_URI_LIST + ";class=java.lang.String", + ClipboardContent.MIME_URI_LIST); + FakeTransferable t = new FakeTransferable() + .add(uriList, new File("/tmp/a.txt").toURI() + "\r\n" + new File("/tmp/b.txt").toURI() + "\r\n"); + + ClipboardContent content = JavaSENativeDragAndDrop.contentFor(t, t.getTransferDataFlavors(), true); + assertArrayEquals(new String[]{"/tmp/a.txt", "/tmp/b.txt"}, content.getFiles(), + "a drop out of a Linux file manager only offers uri-list, and still has to yield files"); + } + + @Test + void theRichestRepresentationOfOneMimeTypeWins() throws Exception { + DataFlavor htmlString = new DataFlavor(ClipboardContent.MIME_HTML + ";class=java.lang.String", + ClipboardContent.MIME_HTML); + DataFlavor htmlStream = new DataFlavor(ClipboardContent.MIME_HTML + ";class=java.io.InputStream", + ClipboardContent.MIME_HTML); + FakeTransferable t = new FakeTransferable() + .add(htmlString, "first") + .add(htmlStream, new ByteArrayInputStream("second".getBytes("UTF-8"))); + + ClipboardContent content = JavaSENativeDragAndDrop.contentFor(t, t.getTransferDataFlavors(), true); + assertEquals("first", content.getText(ClipboardContent.MIME_HTML), + "AWT lists flavors in the source's preference order, so the first one is the answer"); + } + + @Test + void actionsMapBothWays() { + assertEquals(NativeDragOperation.ACTION_COPY, + JavaSENativeDragAndDrop.fromAwtActions( + JavaSENativeDragAndDrop.toAwtActions(NativeDragOperation.ACTION_COPY))); + int all = NativeDragOperation.ACTION_COPY | NativeDragOperation.ACTION_MOVE + | NativeDragOperation.ACTION_LINK; + assertEquals(all, JavaSENativeDragAndDrop.fromAwtActions(JavaSENativeDragAndDrop.toAwtActions(all))); + assertEquals(NativeDragOperation.ACTION_COPY, JavaSENativeDragAndDrop.preferred(all), + "a copy is preferred because it cannot destroy the source's data"); + assertEquals(NativeDragOperation.ACTION_NONE, + JavaSENativeDragAndDrop.preferred(NativeDragOperation.ACTION_NONE)); + } +} From d75db2379b468997fd341114e47a539581092722 Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 07:37:08 +0300 Subject: [PATCH 02/26] Three things CI caught: a stolen tap, a broken watch build, a forbidden modifier Every one of these was invisible to the checks I ran before pushing, and two of them are the kind that would have shipped. **A tap that never arrived.** Installing UIDragInteraction on the Codename One surface unconditionally cost the iOS input-validation suite its tap: drag and long press still worked, tap timed out. UIKit recognizes the drag gesture with a recognizer on the view, and having one there changes how every touch on that view is delivered -- so an application that never drags anything was paying for a gesture it does not use, in the one currency that matters. Both interactions are now attached on demand. Component tells the port when the application marks its first native drag source or drop target (nativeDragSourceRegistered / nativeDropTargetRegistered), and the iOS port attaches the matching interaction then. An application that never asks keeps exactly the input handling it had, which is the whole of what the suite was telling us. The drop half is withheld on the same principle rather than on measurement; it is not known to have been implicated. **A header that reached watchOS.** CN1DragAndDrop.h named CN1View unconditionally, and CN1AppleUI.h deliberately leaves that alias undefined on watchOS -- WatchKit draws through WKInterface objects and there is nothing a CN1View could be there. Every watch build failed on an unknown type name. The declaration now degrades to id on that slice, which is what CN1RenderingView already does with its peer argument and for the same reason. Compile-checked against the iOS, Mac Catalyst, macOS, watchOS and tvOS SDKs, each proved non-vacuous with a deliberate error. **Forbidden PMD rules.** volatile is on the repository's forbidden list and the new router had six of them, plus an unnecessary interface modifier and three anonymous run() methods without @Override. The shared state is now behind one lock, held only across field access and never across a call out -- which is the same rule the threading design already had for its own reasons. Restructuring pressedOn so it installs what a press staged in one unconditional write, rather than clearing and filling in later, also settles the LI_LAZY_INIT_STATIC that the first attempt at this traded the PMD finding for. The lesson for next time is in the middle of that list: I ran SpotBugs locally but not generate-quality-report.py, which is the thing that actually gates PMD. Running it locally now reproduces the failure and the fix, and a probe confirms it is not vacuous. Co-Authored-By: Claude Opus 5 (1M context) --- .../impl/CodenameOneImplementation.java | 26 ++ .../codename1/ui/ClipboardDataProvider.java | 2 +- .../src/com/codename1/ui/Component.java | 13 +- .../com/codename1/ui/NativeDragAndDrop.java | 351 +++++++++++------- .../android/AndroidNativeDragAndDrop.java | 83 +++-- .../impl/javase/JavaSENativeDragAndDrop.java | 40 +- .../com/codename1/impl/javase/JavaSEPort.java | 2 +- Ports/iOSPort/nativeSources/CN1DragAndDrop.h | 25 +- Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 79 +++- Ports/iOSPort/nativeSources/IOSNative.m | 8 + .../codename1/impl/ios/IOSImplementation.java | 11 + .../src/com/codename1/impl/ios/IOSNative.java | 14 + .../NativeDragAndDropSample.java | 1 + .../advancedtopics/NativeDragAndDropDemo.java | 1 + .../TestCodenameOneImplementation.java | 25 ++ .../codename1/ui/NativeDragAndDropTest.java | 30 ++ 16 files changed, 527 insertions(+), 184 deletions(-) diff --git a/CodenameOne/src/com/codename1/impl/CodenameOneImplementation.java b/CodenameOne/src/com/codename1/impl/CodenameOneImplementation.java index 56aeff8335c..3e2c59d4cbf 100644 --- a/CodenameOne/src/com/codename1/impl/CodenameOneImplementation.java +++ b/CodenameOne/src/com/codename1/impl/CodenameOneImplementation.java @@ -5637,6 +5637,32 @@ public boolean startNativeDrag(NativeDragOperation op) { public void cancelNativeDrag() { } + /// Notifies the port that the application has a component that can be dragged through the + /// operating system. + /// + /// A platform whose drag gesture is its own -- UIKit's is -- recognizes that gesture with a + /// recognizer installed on the surface, and installing one changes how every touch on that + /// surface is delivered. Doing it unconditionally would alter touch handling for every + /// application, including the overwhelming majority that never drags anything; a port that + /// needs a recognizer therefore installs it here, the first time an application says it + /// wants one. + /// + /// Called on the event dispatch thread, possibly many times; a port must make it idempotent. + public void nativeDragSourceRegistered() { + } + + /// Notifies the port that the application has a component that accepts drops from the + /// operating system. + /// + /// The counterpart of `#nativeDragSourceRegistered()`, for the same reason: a port attaches + /// whatever the platform needs in order to receive drops only for applications that asked + /// to receive them, so an application that never does keeps exactly the input handling it + /// had. + /// + /// Called on the event dispatch thread, possibly many times; a port must make it idempotent. + public void nativeDropTargetRegistered() { + } + /// True when the port needs the drag image at the moment the operation is staged rather /// than when the drag begins. /// diff --git a/CodenameOne/src/com/codename1/ui/ClipboardDataProvider.java b/CodenameOne/src/com/codename1/ui/ClipboardDataProvider.java index 0d6d528f213..dce7efccbc1 100644 --- a/CodenameOne/src/com/codename1/ui/ClipboardDataProvider.java +++ b/CodenameOne/src/com/codename1/ui/ClipboardDataProvider.java @@ -50,5 +50,5 @@ public interface ClipboardDataProvider { /// /// the value, normally a `String`, a `String[]` of file paths or a `byte[]`, or null when /// the representation turned out to be unavailable - public Object getClipboardData(String mimeType); + Object getClipboardData(String mimeType); } diff --git a/CodenameOne/src/com/codename1/ui/Component.java b/CodenameOne/src/com/codename1/ui/Component.java index 9bc4437239f..195d22f12a0 100644 --- a/CodenameOne/src/com/codename1/ui/Component.java +++ b/CodenameOne/src/com/codename1/ui/Component.java @@ -6517,6 +6517,12 @@ public boolean isNativeDragSource() { /// - `nativeDragSource`: true to hand drags on this component to the operating system public void setNativeDragSource(boolean nativeDragSource) { this.nativeDragSource = nativeDragSource; + if (nativeDragSource) { + // Tells the port that this application wants to drag. Ports whose platform needs a + // gesture recognizer on the surface install it here rather than at startup, so an + // application that never drags keeps exactly the touch handling it has today. + Display.impl.nativeDragSourceRegistered(); + } } /// Returns the operation this component drags, or null when it supplies one per press by @@ -6538,7 +6544,7 @@ public NativeDragOperation getNativeDragOperation() { /// - `nativeDragOperation`: what to drag, or null to stop being a drag source public void setNativeDragOperation(NativeDragOperation nativeDragOperation) { this.nativeDragOperation = nativeDragOperation; - this.nativeDragSource = nativeDragOperation != null; + setNativeDragSource(nativeDragOperation != null); } /// Produces the operation for a drag starting at the given position, invoked on the event @@ -6582,6 +6588,11 @@ public boolean isNativeDropTarget() { /// - `nativeDropTarget`: true to accept operating system drops public void setNativeDropTarget(boolean nativeDropTarget) { this.nativeDropTarget = nativeDropTarget; + if (nativeDropTarget) { + // As with setNativeDragSource: the port attaches whatever the platform needs to + // receive drops only once an application says it wants them. + Display.impl.nativeDropTargetRegistered(); + } } /// Returns the MIME types this component accepts, or null when it accepts anything. diff --git a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java index b31a400a019..2b350c09bc2 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java @@ -73,36 +73,45 @@ /// into another application beside it, and a phone in full screen has nowhere for a drag to go /// even though drags within the application still work. Where nothing is supported the calls /// here are harmless no-ops and the lightweight drag and drop is unaffected. +/// +/// #### Threading +/// +/// The gesture half runs on the event dispatch thread; the receiving half is called from +/// whatever thread the platform hands the port. All of the shared state below is therefore +/// guarded by one lock, and no callback into component or port code is ever made while holding +/// it -- the framework's own event dispatch thread blocks on the platform's UI thread to paint +/// on some ports, so a lock held across a callback is a deadlock waiting for the first drag. public final class NativeDragAndDrop { /// A press further than this from where it started is a drag rather than a click. Measured /// in millimetres so it is a finger on a phone and a pointer on a desktop. private static final float DRAG_THRESHOLD_MM = 1.5f; + /// Guards every field below. Held only across field access, never across a call out. + private static final Object LOCK = new Object(); + /// The operation prepared by the press that is currently down, waiting to see whether the - /// user drags. - /// - /// Written on the event dispatch thread. Read from a native drag thread as well, because a - /// platform that owns the drag gesture itself -- iOS and iPadOS do -- announces the session - /// it started through `#dragSessionStarted()` and this is the operation it started. - private static volatile NativeDragOperation pending; - private static volatile Component pendingSource; + /// user drags, and the component it came from. + private static NativeDragOperation pending; + private static Component pendingSource; + + /// Where that press landed, which is both the drag threshold's origin and the point the + /// drag image is grabbed by. private static int pressX; private static int pressY; + /// Set once this press has been offered to the port, so a platform that declined to start /// the session is not asked again on every drag event of the same gesture. private static boolean startOffered; - /// The session the operating system is currently running, or null. Written on the event - /// dispatch thread and read from the native drag thread, hence volatile. - private static volatile NativeDragOperation active; + /// The session the operating system is currently running, or null. + private static NativeDragOperation active; - /// The drop target the drag is currently over, and the action it last agreed to. Both are - /// read and written from the native drag thread; see the note on - /// `#dragOver(int, int, int, com.codename1.ui.ClipboardContent, int)` about why the answer - /// given to the operating system is the previous callback's. - private static volatile Component currentTarget; - private static volatile int currentAction = NativeDragOperation.ACTION_NONE; - private static volatile boolean overDispatchPending; + /// The drop target the drag is currently over, and the action it last agreed to. See the + /// note on `#dragOver(int, int, int, com.codename1.ui.ClipboardContent, int)` about why the + /// answer given to the operating system is the previous callback's. + private static Component currentTarget; + private static int currentAction = NativeDragOperation.ACTION_NONE; + private static boolean overDispatchPending; private NativeDragAndDrop() { } @@ -148,9 +157,11 @@ public static boolean startDrag(Component source, NativeDragOperation op) { return false; } op.setSource(source); - active = op; - currentTarget = null; - currentAction = NativeDragOperation.ACTION_NONE; + synchronized (LOCK) { + active = op; + currentTarget = null; + currentAction = NativeDragOperation.ACTION_NONE; + } boolean started = false; try { started = Display.impl.startNativeDrag(op); @@ -160,7 +171,11 @@ public static boolean startDrag(Component source, NativeDragOperation op) { Log.e(err); } if (!started) { - active = null; + synchronized (LOCK) { + if (active == op) { // NOPMD CompareObjectsWithEquals + active = null; + } + } } return started; } @@ -175,21 +190,26 @@ public static boolean startDrag(Component source, NativeDragOperation op) { /// the operation the session is carrying, or null when nothing was prepared -- in which case /// the port should refuse to start a session public static NativeDragOperation dragSessionStarted() { - NativeDragOperation op = pending; - if (op == null) { - return null; + NativeDragOperation op; + final Component source; + synchronized (LOCK) { + op = pending; + if (op == null) { + return null; + } + source = pendingSource; + pending = null; + pendingSource = null; + active = op; + currentTarget = null; + currentAction = NativeDragOperation.ACTION_NONE; } - final Component source = pendingSource; - pending = null; - pendingSource = null; - active = op; - currentTarget = null; - currentAction = NativeDragOperation.ACTION_NONE; if (source != null) { // On the event dispatch thread, because it repaints. A component that is draggable // as well as a native drag source would otherwise be left mid-drag with its image // stranded, since the platform stops delivering pointer drags once it takes over. Display.getInstance().callSerially(new Runnable() { + @Override public void run() { source.cancelLightweightDrag(); } @@ -202,7 +222,9 @@ public void run() { /// null when it is not dragging. A drop target uses this to tell a drag it started itself /// from one that arrived from elsewhere, which `NativeDropEvent#isLocal()` reports. public static NativeDragOperation getActiveDrag() { - return active; + synchronized (LOCK) { + return active; + } } // ------------------------------------------------------------------------------------ @@ -216,55 +238,78 @@ public static NativeDragOperation getActiveDrag() { /// source that was pressed and released from being dragged by a later gesture somewhere /// else. static void pressedOn(Component cmp, int x, int y) { - if (pending != null && x == pressX && y == pressY) { - // The same press, dispatched a second time. A top level primes drag and drop on the - // component under the pointer and then again on its nearest draggable ancestor, and - // the ancestor walk below would not find a drag source that sits *between* the two - // -- so clearing here would throw away what the first call correctly staged. Every - // release clears the pending operation, so a later press cannot land on a stale one - // even at the very same pixel. - return; - } - pending = null; - pendingSource = null; - startOffered = false; - if (cmp == null || !isSupported()) { + if (isStagedFor(x, y)) { return; } + // Everything that can call out -- into the component for its payload and its drag + // image, and into the port -- happens outside the lock, and what this press staged is + // then installed in one go. Installing it unconditionally, rather than clearing first + // and filling in later, is also what keeps the two writes from reading as a botched + // lazy initialization of a static field. + NativeDragOperation op = null; Component source = cmp; - while (source != null && !source.isNativeDragSource()) { - source = source.getParent(); + if (cmp != null && isSupported()) { + while (source != null && !source.isNativeDragSource()) { + source = source.getParent(); + } + if (source != null) { + try { + op = source.createNativeDragOperation(x, y); + } catch (Throwable err) { + Log.e(err); + } + } } - if (source == null) { - return; + if (op != null && op.getAllowedActions() == NativeDragOperation.ACTION_NONE) { + op = null; } - NativeDragOperation op; - try { - op = source.createNativeDragOperation(x, y); - } catch (Throwable err) { - Log.e(err); - return; - } - if (op == null || op.getAllowedActions() == NativeDragOperation.ACTION_NONE) { - return; + if (op != null) { + op.setSource(source); + try { + if (op.getDragImage() == null && Display.impl.isNativeDragImageNeededOnPrepare()) { + // The platform asks for the preview from inside its own gesture callback, + // which is not a moment at which a component can be rendered. Rendering here + // costs a snapshot per press on a drag source, which is what the lightweight + // drag has always cost when one starts. + op.setDragImage(source.getDragImage()); + op.setDragImageOffset(x - source.getAbsoluteX(), y - source.getAbsoluteY()); + } + } catch (Throwable err) { + Log.e(err); + } } - op.setSource(source); - pending = op; - pendingSource = source; - pressX = x; - pressY = y; - try { - if (op.getDragImage() == null && Display.impl.isNativeDragImageNeededOnPrepare()) { - // The platform asks for the preview from inside its own gesture callback, which - // is not a moment at which a component can be rendered. Rendering here costs a - // snapshot per press on a drag source, which is what the lightweight drag has - // always cost when one starts. - op.setDragImage(source.getDragImage()); - op.setDragImageOffset(x - source.getAbsoluteX(), y - source.getAbsoluteY()); + stage(op, source, x, y); + if (op != null) { + try { + Display.impl.prepareNativeDrag(op); + } catch (Throwable err) { + Log.e(err); } - Display.impl.prepareNativeDrag(op); - } catch (Throwable err) { - Log.e(err); + } + } + + /// True when this exact press has already staged an operation. + /// + /// A top level primes drag and drop on the component under the pointer and then again on + /// its nearest draggable ancestor, and the ancestor walk in `#pressedOn(Component, int, + /// int)` would not find a drag source that sits *between* the two -- so restaging would + /// throw away what the first call correctly staged. Every release clears the pending + /// operation, so a later press cannot land on a stale one even at the very same pixel. + private static boolean isStagedFor(int x, int y) { + synchronized (LOCK) { + return pending != null && x == pressX && y == pressY; + } + } + + /// Installs what a press staged, or clears it when the press staged nothing. Unconditional + /// rather than a clear followed by a fill, so that one press leaves one consistent state. + private static void stage(NativeDragOperation op, Component source, int x, int y) { + synchronized (LOCK) { + pending = op; + pendingSource = op == null ? null : source; + pressX = x; + pressY = y; + startOffered = false; } } @@ -282,32 +327,40 @@ static void pressedOn(Component cmp, int x, int y) { /// true when the native drag has taken the gesture over and the framework should not also /// treat it as a scroll or a lightweight drag static boolean pointerDragged(int x, int y) { - NativeDragOperation op = pending; - if (op == null) { - // A session already running owns the gesture. Ports differ on whether they keep - // delivering pointer drags during a native drag; swallowing them here means the - // ones that do cannot scroll the surface out from under the drag. - return active != null; - } int threshold = dragThreshold(); - if (Math.abs(x - pressX) < threshold && Math.abs(y - pressY) < threshold) { - return false; - } - if (startOffered) { - // Already offered for this gesture and not taken, which is what a platform that - // starts the session on its own recognizer looks like. Leave the gesture alone - // until that recognizer fires; it announces itself through dragSessionStarted(). - return active != null; + NativeDragOperation op; + Component source; + int grabX; + int grabY; + synchronized (LOCK) { + if (pending == null) { + // A session already running owns the gesture. Ports differ on whether they keep + // delivering pointer drags during a native drag; swallowing them here means the + // ones that do cannot scroll the surface out from under the drag. + return active != null; + } + if (Math.abs(x - pressX) < threshold && Math.abs(y - pressY) < threshold) { + return false; + } + if (startOffered) { + // Already offered for this gesture and not taken, which is what a platform that + // starts the session on its own recognizer looks like. Leave the gesture alone + // until that recognizer fires; it announces itself through dragSessionStarted(). + return active != null; + } + startOffered = true; + op = pending; + source = pendingSource; + grabX = pressX; + grabY = pressY; } - startOffered = true; - Component source = pendingSource; if (op.getDragImage() == null && source != null) { try { op.setDragImage(source.getDragImage()); // Only when the image is the one we just rendered from the component. An // application that supplied its own image may also have positioned it, and // overwriting that offset would tear the image away from the pointer. - op.setDragImageOffset(pressX - source.getAbsoluteX(), pressY - source.getAbsoluteY()); + op.setDragImageOffset(grabX - source.getAbsoluteX(), grabY - source.getAbsoluteY()); } catch (Throwable err) { Log.e(err); } @@ -319,8 +372,12 @@ static boolean pointerDragged(int x, int y) { // in the first place, so there is nothing to keep. return false; } - pending = null; - pendingSource = null; + synchronized (LOCK) { + if (pending == op) { // NOPMD CompareObjectsWithEquals + pending = null; + pendingSource = null; + } + } if (source != null) { // A component can be both draggable and a native drag source. The native session // owns the gesture from here, and the port stops delivering pointer drags, so the @@ -334,10 +391,14 @@ static boolean pointerDragged(int x, int y) { /// Drops the operation prepared by a press that turned out to be a click. Called as the /// pointer is released. static void pointerReleased() { - startOffered = false; - if (pending != null) { + boolean hadPending; + synchronized (LOCK) { + startOffered = false; + hadPending = pending != null; pending = null; pendingSource = null; + } + if (hadPending) { try { Display.impl.cancelNativeDrag(); } catch (Throwable err) { @@ -416,28 +477,32 @@ public static int dragEnter(int windowId, int x, int y, ClipboardContent content /// #### Returns /// /// the action a drop would perform right now, or `NativeDragOperation#ACTION_NONE` - public static int dragOver(final int windowId, final int x, final int y, - final ClipboardContent content, final int allowedActions) { + public static int dragOver(int windowId, int x, int y, ClipboardContent content, int allowedActions) { Component target = findTarget(windowId, x, y, content); - Component previous = currentTarget; - if (previous != target) { // NOPMD CompareObjectsWithEquals - currentTarget = target; - currentAction = target == null ? NativeDragOperation.ACTION_NONE - : (allowedActions & target.getAcceptedDropActions()) == 0 - ? NativeDragOperation.ACTION_NONE - : preferredAction(allowedActions & target.getAcceptedDropActions()); + Component previous; + boolean changed; + boolean dispatchOver = false; + int answer; + synchronized (LOCK) { + previous = currentTarget; + changed = previous != target; // NOPMD CompareObjectsWithEquals + if (changed) { + currentTarget = target; + currentAction = target == null ? NativeDragOperation.ACTION_NONE + : preferredAction(allowedActions & target.getAcceptedDropActions()); + } else if (target != null && !overDispatchPending) { + overDispatchPending = true; + dispatchOver = true; + } + answer = target == null ? NativeDragOperation.ACTION_NONE : currentAction; + } + if (changed) { dispatch(previous, ActionEvent.Type.NativeDragExit, content, x, y, allowedActions); dispatch(target, ActionEvent.Type.NativeDragEnter, content, x, y, allowedActions); - return currentAction; - } - if (target == null) { - return NativeDragOperation.ACTION_NONE; - } - if (!overDispatchPending) { - overDispatchPending = true; + } else if (dispatchOver) { dispatch(target, ActionEvent.Type.NativeDragOver, content, x, y, allowedActions); } - return currentAction; + return answer; } /// Reports that a native drag has left the application's surfaces without dropping. @@ -446,9 +511,12 @@ public static int dragOver(final int windowId, final int x, final int y, /// /// - `windowId`: the id of the window the drag left, or zero for the main surface public static void dragExit(int windowId) { - Component previous = currentTarget; - currentTarget = null; - currentAction = NativeDragOperation.ACTION_NONE; + Component previous; + synchronized (LOCK) { + previous = currentTarget; + currentTarget = null; + currentAction = NativeDragOperation.ACTION_NONE; + } dispatch(previous, ActionEvent.Type.NativeDragExit, null, 0, 0, NativeDragOperation.ACTION_NONE); } @@ -477,19 +545,16 @@ public static void dragExit(int windowId) { /// the pointer took the drop and the port should report the transfer as failed public static int drop(int windowId, int x, int y, ClipboardContent content, int action) { Component target = findTarget(windowId, x, y, content); - currentTarget = null; - overDispatchPending = false; - if (target == null) { - currentAction = NativeDragOperation.ACTION_NONE; - return NativeDragOperation.ACTION_NONE; + int accepted = target == null ? NativeDragOperation.ACTION_NONE + : preferredAction(action & target.getAcceptedDropActions()); + synchronized (LOCK) { + currentTarget = null; + overDispatchPending = false; + currentAction = accepted; } - int accepted = action & target.getAcceptedDropActions(); if (accepted == NativeDragOperation.ACTION_NONE) { - currentAction = NativeDragOperation.ACTION_NONE; return NativeDragOperation.ACTION_NONE; } - accepted = preferredAction(accepted); - currentAction = accepted; dispatch(target, ActionEvent.Type.NativeDrop, content, x, y, accepted); return accepted; } @@ -503,15 +568,19 @@ public static int drop(int windowId, int x, int y, ClipboardContent content, int /// - `performedAction`: the action the receiver performed, or /// `NativeDragOperation#ACTION_NONE` when the drag was cancelled or refused public static void dragCompleted(final int performedAction) { - final NativeDragOperation op = active; - active = null; - currentTarget = null; - currentAction = NativeDragOperation.ACTION_NONE; - overDispatchPending = false; + final NativeDragOperation op; + synchronized (LOCK) { + op = active; + active = null; + currentTarget = null; + currentAction = NativeDragOperation.ACTION_NONE; + overDispatchPending = false; + } if (op == null) { return; } Display.getInstance().callSerially(new Runnable() { + @Override public void run() { op.fireCompleted(performedAction); } @@ -588,12 +657,20 @@ private static void dispatch(final Component target, final ActionEvent.Type type final ClipboardContent content, final int x, final int y, final int allowedActions) { if (target == null) { if (type == ActionEvent.Type.NativeDragOver) { - overDispatchPending = false; + synchronized (LOCK) { + overDispatchPending = false; + } } return; } - final boolean local = active != null; + final boolean local; + final int startingAction; + synchronized (LOCK) { + local = active != null; + startingAction = currentAction; + } Display.getInstance().callSerially(new Runnable() { + @Override public void run() { try { NativeDropEvent ev = new NativeDropEvent(target, type, content, x, y, allowedActions, local); @@ -601,19 +678,23 @@ public void run() { // The target starts from what the framework already agreed to, so a // target that does not care keeps the answer stable instead of // resetting it to the default on every event. - ev.accept(currentAction); + ev.accept(startingAction); } target.dispatchNativeDropEvent(ev); if (type == ActionEvent.Type.NativeDragOver || type == ActionEvent.Type.NativeDragEnter) { - if (currentTarget == target) { // NOPMD CompareObjectsWithEquals - currentAction = ev.getAcceptedAction(); + synchronized (LOCK) { + if (currentTarget == target) { // NOPMD CompareObjectsWithEquals + currentAction = ev.getAcceptedAction(); + } } } } catch (Throwable err) { Log.e(err); } finally { if (type == ActionEvent.Type.NativeDragOver) { - overDispatchPending = false; + synchronized (LOCK) { + overDispatchPending = false; + } } } } diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java b/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java index 8fbc79b30bb..a21195385f8 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java @@ -54,15 +54,42 @@ /// reader serves both. A URI from another application is only readable while the drop's /// permission grant is held, which is why the content is read inside the drop callback rather /// than handed to the event dispatch thread to read later. -class AndroidNativeDragAndDrop { +final class AndroidNativeDragAndDrop { /// The operation currently being dragged out of this application, so that the outcome - /// reported when the drag ends can be attributed to it. + /// reported when the drag ends can be attributed to it, and the action last agreed with the + /// framework, reported back when the drop is accepted -- Android's drag events carry no + /// copy/move/link distinction of their own. + /// + /// Both are written from the Codename One event dispatch thread and read from the Android + /// UI thread, so the lock is what publishes one to the other. + private static final Object LOCK = new Object(); private static NativeDragOperation exporting; - - /// The action last agreed with the framework, reported back when the drop is accepted. - /// Android's drag events carry no copy/move/link distinction of their own. private static int lastAction = NativeDragOperation.ACTION_NONE; + private static NativeDragOperation exporting() { + synchronized (LOCK) { + return exporting; + } + } + + private static void setExporting(NativeDragOperation op) { + synchronized (LOCK) { + exporting = op; + } + } + + private static int lastAction() { + synchronized (LOCK) { + return lastAction; + } + } + + private static void setLastAction(int action) { + synchronized (LOCK) { + lastAction = action; + } + } + private AndroidNativeDragAndDrop() { } @@ -90,8 +117,9 @@ static void install(final AndroidImplementation impl, final View view) { } try { view.setOnDragListener(new View.OnDragListener() { + @Override public boolean onDrag(View v, DragEvent event) { - return handle(impl, v, event); + return handle(impl, event); } }); } catch (Throwable err) { @@ -115,9 +143,10 @@ static boolean startDrag(final AndroidImplementation impl, final NativeDragOpera if (clip == null) { return false; } - exporting = op; - lastAction = NativeDragOperation.ACTION_NONE; + setExporting(op); + setLastAction(NativeDragOperation.ACTION_NONE); view.post(new Runnable() { + @Override public void run() { boolean started = false; try { @@ -132,7 +161,7 @@ public void run() { Log.e(err); } if (!started) { - exporting = null; + setExporting(null); NativeDragAndDrop.dragCompleted(NativeDragOperation.ACTION_NONE); } } @@ -142,12 +171,12 @@ public void run() { /// Forgets a prepared operation because the press turned out to be a click. static void cancelDrag() { - exporting = null; + setExporting(null); } // ------------------------------------------------------------------------------------ - private static boolean handle(AndroidImplementation impl, View view, DragEvent event) { + private static boolean handle(AndroidImplementation impl, DragEvent event) { try { switch (event.getAction()) { case DragEvent.ACTION_DRAG_STARTED: @@ -156,26 +185,27 @@ private static boolean handle(AndroidImplementation impl, View view, DragEvent e // is unconditional and the real filtering happens per position below. return true; case DragEvent.ACTION_DRAG_ENTERED: - lastAction = NativeDragAndDrop.dragEnter(0, (int) event.getX(), (int) event.getY(), - describe(event.getClipDescription()), allowedActions()); + setLastAction(NativeDragAndDrop.dragEnter(0, (int) event.getX(), (int) event.getY(), + describe(event.getClipDescription()), allowedActions())); return true; case DragEvent.ACTION_DRAG_LOCATION: - lastAction = NativeDragAndDrop.dragOver(0, (int) event.getX(), (int) event.getY(), - describe(event.getClipDescription()), allowedActions()); + setLastAction(NativeDragAndDrop.dragOver(0, (int) event.getX(), (int) event.getY(), + describe(event.getClipDescription()), allowedActions())); return true; case DragEvent.ACTION_DRAG_EXITED: NativeDragAndDrop.dragExit(0); - lastAction = NativeDragOperation.ACTION_NONE; + setLastAction(NativeDragOperation.ACTION_NONE); return true; case DragEvent.ACTION_DROP: - return drop(impl, view, event); + return drop(impl, event); case DragEvent.ACTION_DRAG_ENDED: - if (exporting != null) { - exporting = null; + if (exporting() != null) { + int allowed = allowedActions(); + setExporting(null); NativeDragAndDrop.dragCompleted(event.getResult() - ? preferred(allowedActions()) : NativeDragOperation.ACTION_NONE); + ? preferred(allowed) : NativeDragOperation.ACTION_NONE); } - lastAction = NativeDragOperation.ACTION_NONE; + setLastAction(NativeDragOperation.ACTION_NONE); return true; default: return false; @@ -186,7 +216,7 @@ private static boolean handle(AndroidImplementation impl, View view, DragEvent e } } - private static boolean drop(AndroidImplementation impl, View view, DragEvent event) { + private static boolean drop(AndroidImplementation impl, DragEvent event) { // A URI dropped by another application is only readable while this grant is held, and // the grant only exists from here on. Reading the content inside this method rather // than on the event dispatch thread is what keeps a dropped file readable. @@ -199,10 +229,10 @@ private static boolean drop(AndroidImplementation impl, View view, DragEvent eve } } ClipboardContent content = impl.contentFromClip(event.getClipData()); - int action = lastAction == NativeDragOperation.ACTION_NONE - ? preferred(allowedActions()) : lastAction; + int action = lastAction() == NativeDragOperation.ACTION_NONE + ? preferred(allowedActions()) : lastAction(); int accepted = NativeDragAndDrop.drop(0, (int) event.getX(), (int) event.getY(), content, action); - lastAction = NativeDragOperation.ACTION_NONE; + setLastAction(NativeDragOperation.ACTION_NONE); return accepted != NativeDragOperation.ACTION_NONE; } @@ -210,7 +240,7 @@ private static boolean drop(AndroidImplementation impl, View view, DragEvent eve /// one arriving from another application is a copy, because Android's cross-application /// drag has no way to express anything else. private static int allowedActions() { - NativeDragOperation op = exporting; + NativeDragOperation op = exporting(); return op == null ? NativeDragOperation.ACTION_COPY : op.getAllowedActions(); } @@ -260,6 +290,7 @@ private static void declare(ClipboardContent content, String mime) { return; } content.setDataProvider(mime, new ClipboardDataProvider() { + @Override public Object getClipboardData(String requested) { // Android reveals nothing until the drop; the drop callback replaces this with // the real content. diff --git a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java index 5fc6a77f68e..23ba42c4a9e 100644 --- a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java +++ b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java @@ -80,8 +80,22 @@ final class JavaSENativeDragAndDrop { /// The operation the Codename One event thread has asked to export, read by the transfer /// handler on the AWT thread when the drag actually starts. One process drags one thing at - /// a time, so a single slot is the whole of the state. - private static volatile NativeDragOperation exporting; + /// a time, so a single slot is the whole of the state; the lock is what publishes it from + /// one thread to the other. + private static final Object LOCK = new Object(); + private static NativeDragOperation exporting; + + private static NativeDragOperation exporting() { + synchronized (LOCK) { + return exporting; + } + } + + private static void setExporting(NativeDragOperation op) { + synchronized (LOCK) { + exporting = op; + } + } private JavaSENativeDragAndDrop() { } @@ -130,8 +144,9 @@ static boolean startDrag(final JavaSEPort port, final NativeDragOperation op) { final Point offset = new Point( (int) (op.getDragImageOffsetX() / target.canvasScale()), (int) (op.getDragImageOffsetY() / target.canvasScale())); - exporting = op; + setExporting(op); EventQueue.invokeLater(new Runnable() { + @Override public void run() { try { TransferHandler handler = target.getTransferHandler(); @@ -146,7 +161,7 @@ public void run() { handler.exportAsDrag(target, trigger, toAwtAction(preferred(op.getAllowedActions()))); } catch (Throwable err) { Log.e(err); - exporting = null; + setExporting(null); NativeDragAndDrop.dragCompleted(NativeDragOperation.ACTION_NONE); } } @@ -155,8 +170,8 @@ public void run() { } /// Forgets a prepared operation because the press turned out to be a click. - static void cancelDrag(JavaSEPort port) { - exporting = null; + static void cancelDrag() { + setExporting(null); } /// Renders the operation's drag image at the size AWT expects. @@ -307,6 +322,7 @@ static ClipboardContent contentFor(final Transferable transferable, DataFlavor[] } } else { content.setDataProvider(mime, new ClipboardDataProvider() { + @Override public Object getClipboardData(String requested) { return readValue(transferable, flavor, requested); } @@ -324,6 +340,7 @@ public Object getClipboardData(String requested) { } else { final ClipboardContent describing = content; content.setDataProvider(ClipboardContent.MIME_FILE, new ClipboardDataProvider() { + @Override public Object getClipboardData(String requested) { String[] paths = pathsFromUriList(describing.getText(ClipboardContent.MIME_URI_LIST)); if (paths == null) { @@ -445,19 +462,19 @@ private static byte[] readBytes(InputStream input) throws Exception { private static final class Cn1TransferHandler extends TransferHandler { @Override public int getSourceActions(JComponent c) { - NativeDragOperation op = exporting; + NativeDragOperation op = exporting(); return op == null ? NONE : toAwtActions(op.getAllowedActions()); } @Override protected Transferable createTransferable(JComponent c) { - NativeDragOperation op = exporting; + NativeDragOperation op = exporting(); return op == null ? null : new JavaSEPort.RichTransferable(op.getContent()); } @Override protected void exportDone(JComponent source, Transferable data, int action) { - exporting = null; + setExporting(null); NativeDragAndDrop.dragCompleted(preferred(fromAwtActions(action))); } } @@ -470,18 +487,22 @@ private static final class Cn1DropTargetListener implements DropTargetListener { this.canvas = canvas; } + @Override public void dragEnter(DropTargetDragEvent e) { respond(e, true); } + @Override public void dragOver(DropTargetDragEvent e) { respond(e, false); } + @Override public void dropActionChanged(DropTargetDragEvent e) { respond(e, false); } + @Override public void dragExit(DropTargetEvent e) { try { NativeDragAndDrop.dragExit(canvas.windowId); @@ -490,6 +511,7 @@ public void dragExit(DropTargetEvent e) { } } + @Override public void drop(DropTargetDropEvent e) { try { int allowed = fromAwtActions(e.getSourceActions()); diff --git a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSEPort.java b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSEPort.java index fab4064a4e9..62f42fc8753 100644 --- a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSEPort.java +++ b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSEPort.java @@ -1913,7 +1913,7 @@ public boolean startNativeDrag(com.codename1.ui.NativeDragOperation op) { @Override public void cancelNativeDrag() { - JavaSENativeDragAndDrop.cancelDrag(this); + JavaSENativeDragAndDrop.cancelDrag(); } @Override diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.h b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h index 9c0d000a1ed..48da8d91918 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.h +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h @@ -54,8 +54,31 @@ BOOL CN1DragAndDropSupported(void); /// application on screen to drop it into. BOOL CN1DragOutsideAppSupported(void); -/// Attaches the drag and the drop interactions to the Codename One surface. +/// Remembers the Codename One surface, so the interactions can be attached to it later. +/// +/// Neither interaction is attached here. UIDragInteraction recognizes its gesture with a +/// recognizer installed on the view, and installing one changes how every touch on that view is +/// delivered -- a plain tap stopped reaching the framework at all. The overwhelming majority of +/// applications never drag anything and must not pay for that; the drop half is withheld on the +/// same principle rather than on measurement. +/// +/// The parameter degrades to `id` on watchOS, which has no CN1View at all -- WatchKit draws +/// through WKInterface objects and CN1AppleUI.h deliberately leaves the alias undefined there. +/// CN1RenderingView's peer argument does the same thing for the same reason. Naming the type +/// unconditionally broke every watch build, since this header reaches that slice too. +#if TARGET_OS_WATCH +void CN1InstallDragAndDrop(id view); +#else void CN1InstallDragAndDrop(CN1View* view); +#endif + +/// Attaches the drag interaction, because the application has a component that can be dragged +/// out. Idempotent, and safe to call from any thread. +void CN1EnableNativeDragSource(void); + +/// Attaches the drop interaction, because the application has a component that accepts drops. +/// Idempotent, and safe to call from any thread. +void CN1EnableNativeDropTarget(void); /// Stages the drag a press has made possible: which representations it can offer, what the /// receiver may do with them, and the image to show under the finger. The representations are diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index 0631a43eba6..c791d1a9ee9 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -41,8 +41,19 @@ BOOL CN1DragOutsideAppSupported(void) { return NO; } +#if TARGET_OS_WATCH +void CN1InstallDragAndDrop(id view) { +} +#else void CN1InstallDragAndDrop(CN1View* view) { } +#endif + +void CN1EnableNativeDragSource(void) { +} + +void CN1EnableNativeDropTarget(void) { +} void CN1PrepareNativeDrag(NSString* mimeTypes, int allowedActions, NSData* dragImagePng, int touchX, int touchY) { @@ -80,6 +91,12 @@ void CN1CancelNativeDrag(void) { /// True while this application is the source of the session in progress. static BOOL cn1DraggingOut = NO; +/// The surface, remembered so the drag interaction can be attached later, and the delegate that +/// serves both interactions. The delegate outlives the surface, which lives for the life of the +/// process, and the interactions hold it weakly. +static CN1View* cn1DragSurface = nil; +static id cn1DragDelegate = nil; + /// The MIME types the framework names, mapped onto the uniform type identifiers UIKit and /// every other application on the system speak. static NSString* cn1UtiForMime(NSString* mime) { @@ -501,21 +518,63 @@ void CN1InstallDragAndDrop(CN1View* view) { return; } if (@available(iOS 11.0, *)) { - CN1DragAndDropDelegate* delegate = [[CN1DragAndDropDelegate alloc] init]; - UIDragInteraction* drag = [[UIDragInteraction alloc] initWithDelegate:delegate]; - // Without this a drag never begins on iPhone: UIKit enables drag interactions on iPad - // by default and leaves them off elsewhere. - drag.enabled = YES; - [view addInteraction:drag]; - UIDropInteraction* drop = [[UIDropInteraction alloc] initWithDelegate:delegate]; - [view addInteraction:drop]; + cn1DragSurface = view; // The delegate is deliberately not released: the interactions hold their delegate // weakly and it has to outlive the surface, which lives for the life of the process. + cn1DragDelegate = [[CN1DragAndDropDelegate alloc] init]; + } +} + +/// True when an interaction of this class is already on the surface. Both enable calls run for +/// every component an application marks, which for a list is every row. +static BOOL cn1HasInteraction(Class kind) { + for (id existing in cn1DragSurface.interactions) { + if ([existing isKindOfClass:kind]) { + return YES; + } + } + return NO; +} + +void CN1EnableNativeDragSource(void) { + if (!CN1DragAndDropSupported()) { + return; + } + dispatch_async(dispatch_get_main_queue(), ^{ + if (@available(iOS 11.0, *)) { + if (cn1DragSurface == nil || cn1DragDelegate == nil + || cn1HasInteraction([UIDragInteraction class])) { + return; + } + UIDragInteraction* drag = [[UIDragInteraction alloc] initWithDelegate:cn1DragDelegate]; + // Without this a drag never begins on iPhone: UIKit enables drag interactions on + // iPad by default and leaves them off elsewhere. + drag.enabled = YES; + [cn1DragSurface addInteraction:drag]; #ifndef CN1_USE_ARC - [drag release]; - [drop release]; + [drag release]; #endif + } + }); +} + +void CN1EnableNativeDropTarget(void) { + if (!CN1DragAndDropSupported()) { + return; } + dispatch_async(dispatch_get_main_queue(), ^{ + if (@available(iOS 11.0, *)) { + if (cn1DragSurface == nil || cn1DragDelegate == nil + || cn1HasInteraction([UIDropInteraction class])) { + return; + } + UIDropInteraction* drop = [[UIDropInteraction alloc] initWithDelegate:cn1DragDelegate]; + [cn1DragSurface addInteraction:drop]; +#ifndef CN1_USE_ARC + [drop release]; +#endif + } + }); } #endif diff --git a/Ports/iOSPort/nativeSources/IOSNative.m b/Ports/iOSPort/nativeSources/IOSNative.m index e3df8646057..429a17f5d74 100644 --- a/Ports/iOSPort/nativeSources/IOSNative.m +++ b/Ports/iOSPort/nativeSources/IOSNative.m @@ -1099,6 +1099,14 @@ void com_codename1_impl_ios_IOSNative_cancelNativeDrag__(CN1_THREAD_STATE_MULTI_ }); } +void com_codename1_impl_ios_IOSNative_enableNativeDragSource__(CN1_THREAD_STATE_MULTI_ARG JAVA_OBJECT instanceObject) { + CN1EnableNativeDragSource(); +} + +void com_codename1_impl_ios_IOSNative_enableNativeDropTarget__(CN1_THREAD_STATE_MULTI_ARG JAVA_OBJECT instanceObject) { + CN1EnableNativeDropTarget(); +} + int CN1NativeDragDeliverOver(int x, int y, NSString* mimeTypes, int allowedActions, BOOL entering) { return (int)com_codename1_impl_ios_IOSImplementation_nativeDragOverCallback___int_int_java_lang_String_int_boolean_R_int( CN1_THREAD_GET_STATE_PASS_ARG x, y, diff --git a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java index efaabe88f56..6e6d5d90320 100644 --- a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java +++ b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java @@ -9178,6 +9178,16 @@ public void cancelNativeDrag() { nativeInstance.cancelNativeDrag(); } + @Override + public void nativeDragSourceRegistered() { + nativeInstance.enableNativeDragSource(); + } + + @Override + public void nativeDropTargetRegistered() { + nativeInstance.enableNativeDropTarget(); + } + /// Encodes a drag preview as PNG, the one image format the whole bridge speaks. private static byte[] pngBytes(Image image) { if (image == null) { @@ -9236,6 +9246,7 @@ private static ClipboardContent describe(String mimeTypes) { } for (int iter = 0; iter < mimes.length; iter++) { content.setDataProvider(mimes[iter], new ClipboardDataProvider() { + @Override public Object getClipboardData(String mimeType) { return null; } diff --git a/Ports/iOSPort/src/com/codename1/impl/ios/IOSNative.java b/Ports/iOSPort/src/com/codename1/impl/ios/IOSNative.java index 8ac6afa9886..c2fff8ee81a 100644 --- a/Ports/iOSPort/src/com/codename1/impl/ios/IOSNative.java +++ b/Ports/iOSPort/src/com/codename1/impl/ios/IOSNative.java @@ -396,6 +396,20 @@ native void setNativeDragPayload(String plain, String html, String rtf, byte[] i /// Drops whatever `#prepareNativeDrag(java.lang.String, int, byte[], int, int)` staged, /// because the press turned out to be a tap. native void cancelNativeDrag(); + + /// Attaches the drag interaction, because the application has a component that can be + /// dragged out. + /// + /// UIKit recognizes the drag gesture with a recognizer installed on the surface, and having + /// one there changes how every touch is delivered -- with it attached unconditionally a + /// plain tap stopped reaching the framework. So it is attached on demand, and an + /// application that never drags anything keeps exactly the touch handling it had. + /// Idempotent. + native void enableNativeDragSource(); + + /// Attaches the drop interaction, because the application has a component that accepts + /// drops. Withheld until then on the same principle as the drag interaction. Idempotent. + native void enableNativeDropTarget(); native void setPinchToZoomEnabled(long peer, boolean e); native void setNativeBrowserScrollingEnabled(long peer, boolean e); diff --git a/Samples/samples/NativeDragAndDropSample/NativeDragAndDropSample.java b/Samples/samples/NativeDragAndDropSample/NativeDragAndDropSample.java index 30565ad3ee8..74c944fc942 100644 --- a/Samples/samples/NativeDragAndDropSample/NativeDragAndDropSample.java +++ b/Samples/samples/NativeDragAndDropSample/NativeDragAndDropSample.java @@ -123,6 +123,7 @@ private Component fileDragSource() { ClipboardContent content = new ClipboardContent() .setData(ClipboardContent.MIME_TEXT, "codenameone-note.txt") .setDataProvider(ClipboardContent.MIME_FILE, new ClipboardDataProvider() { + @Override public Object getClipboardData(String mimeType) { return writeNote(); } diff --git a/docs/demos/common/src/main/java/com/codenameone/developerguide/advancedtopics/NativeDragAndDropDemo.java b/docs/demos/common/src/main/java/com/codenameone/developerguide/advancedtopics/NativeDragAndDropDemo.java index f631301e297..7dbff1a463d 100644 --- a/docs/demos/common/src/main/java/com/codenameone/developerguide/advancedtopics/NativeDragAndDropDemo.java +++ b/docs/demos/common/src/main/java/com/codenameone/developerguide/advancedtopics/NativeDragAndDropDemo.java @@ -67,6 +67,7 @@ public void showFileDragSource() { ClipboardContent content = new ClipboardContent() .setData(ClipboardContent.MIME_TEXT, "note.txt") .setDataProvider(ClipboardContent.MIME_FILE, new ClipboardDataProvider() { + @Override public Object getClipboardData(String mimeType) { return writeNote(); } diff --git a/maven/core-unittests/src/test/java/com/codename1/testing/TestCodenameOneImplementation.java b/maven/core-unittests/src/test/java/com/codename1/testing/TestCodenameOneImplementation.java index 6c91331057e..d55bf34ea6b 100644 --- a/maven/core-unittests/src/test/java/com/codename1/testing/TestCodenameOneImplementation.java +++ b/maven/core-unittests/src/test/java/com/codename1/testing/TestCodenameOneImplementation.java @@ -123,6 +123,8 @@ public class TestCodenameOneImplementation extends CodenameOneImplementation { private com.codename1.ui.NativeDragOperation startedNativeDrag; private int cancelledNativeDrags; private boolean nativeDragStartRefused; + private int nativeDragSourceRegistrations; + private int nativeDropTargetRegistrations; private final TestFont defaultFont = new TestFont(8, 16); private int displayWidth = 1080; @@ -5540,10 +5542,33 @@ public int getCancelledNativeDrags() { return cancelledNativeDrags; } + @Override + public void nativeDragSourceRegistered() { + nativeDragSourceRegistrations++; + } + + /// How many times the framework has told the port that this application wants to drag. A + /// port whose platform needs a gesture recognizer installs it on the strength of this. + public int getNativeDragSourceRegistrations() { + return nativeDragSourceRegistrations; + } + + @Override + public void nativeDropTargetRegistered() { + nativeDropTargetRegistrations++; + } + + /// How many times the framework has told the port that this application accepts drops. + public int getNativeDropTargetRegistrations() { + return nativeDropTargetRegistrations; + } + /// Forgets everything recorded, so one test does not see another's drag. public void resetNativeDragState() { preparedNativeDrag = null; startedNativeDrag = null; cancelledNativeDrags = 0; + nativeDragSourceRegistrations = 0; + nativeDropTargetRegistrations = 0; } } diff --git a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java index 9fea5fbd932..9aa071e50a7 100644 --- a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java +++ b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java @@ -302,6 +302,36 @@ void dropOnNothingReportsFailureSoThePortCanTellTheSource() { // The gesture: a press on a drag source becomes an operating system drag // ------------------------------------------------------------------------------------ + @FormTest + void becomingADragSourceTellsThePort() { + implementation.resetNativeDragState(); + try { + Container cmp = new Container(); + assertEquals(0, implementation.getNativeDragSourceRegistrations()); + + cmp.setNativeDragOperation(new NativeDragOperation("x")); + assertEquals(1, implementation.getNativeDragSourceRegistrations(), + "a platform that needs a gesture recognizer installs it on the strength of " + + "this, so an application that never drags keeps its touch handling"); + + cmp.setNativeDragSource(true); + assertEquals(2, implementation.getNativeDragSourceRegistrations()); + + cmp.setNativeDragOperation(null); + cmp.setNativeDragSource(false); + assertEquals(2, implementation.getNativeDragSourceRegistrations(), + "giving up on dragging is not a request for a recognizer"); + + assertEquals(0, implementation.getNativeDropTargetRegistrations()); + cmp.setNativeDropTarget(true); + assertEquals(1, implementation.getNativeDropTargetRegistrations()); + cmp.setNativeDropTarget(false); + assertEquals(1, implementation.getNativeDropTargetRegistrations()); + } finally { + implementation.resetNativeDragState(); + } + } + @FormTest void aDragOnANativeDragSourceIsHandedToThePlatform() { implementation.resetNativeDragState(); From f8888cd160bc950ccfe67c14c94172e4ca5856a0 Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 08:25:39 +0300 Subject: [PATCH 03/26] Review: five ways the payload or the outcome was quietly losing something All five findings held up against the code. Every one of them is a case of the bridge narrowing what the framework handed it, and none of them fails loudly. **Android reported every move as a copy.** ACTION_DRAG_ENDED read the allowed actions after clearing the exporting operation, so allowedActions() answered with its copy fallback; and a local drop's real answer had already been thrown away in drop(). A source that offered ACTION_MOVE and deletes its data on completion therefore never did. The action a local drop settled on is now kept until the session ends, and the completion is settled before the operation is forgotten. A drop into another application still reports copy, because Android's drag protocol has no notion of copy versus move and ACTION_DRAG_ENDED carries only a boolean -- that is now stated where the decision is made, along with why copy rather than move is the safe reading of "it worked and we do not know how". **Android advertised only text.** clipDataFor() built a text ClipData and then appended URI items, and ClipData.addItem does not widen the description -- so a clip carrying text *and* a file described itself as text only. A Codename One target filtering on MIME_FILE rejected it and an external receiver could not select the richer representation. The clip is now constructed from the union of its types. This also fixes the same defect on the clipboard, which shares the conversion. **iOS told local drop sessions the source allowed only a copy.** A move-only drag then had no action in common with a move-only target and could not be dropped at all, and a copy-or-move drag could only ever be proposed as a copy, so no in-application reorder could report a move back to its source. A session this application started is now described by the actions it actually allows, taken from the framework at session start. A session from another application is still told copy, because UIKit tells a drop interaction nothing about what the far side permits. **iOS forwarded five representations out of however many were advertised.** prepare advertises everything the content holds, but the payload bridge carried a fixed list, so an operation holding only MIME_MARKDOWN advertised a type it then could not produce -- and a drag that begins with no items is cancelled on the spot. The bridge now takes one representation at a time and the Java side pushes all of them, resolving promised values as it goes. Unmapped MIME types reach the system through UTType, falling back to the MIME type itself as an opaque identifier: unread by a receiver that does not know it, which is a great deal better than dropped. This also stops JPEG bytes being published as PNG. **A reused operation reported the last drag's result.** setNativeDragOperation documents the instance as reusable, so getPerformedAction() went on answering ACTION_MOVE through the whole of the next drag, contradicting its own contract that the value before completion is ACTION_NONE. It is cleared when the operation is installed as the active one. Co-Authored-By: Claude Opus 5 (1M context) --- .../com/codename1/ui/NativeDragAndDrop.java | 2 + .../com/codename1/ui/NativeDragOperation.java | 11 ++ .../impl/android/AndroidImplementation.java | 59 ++++--- .../android/AndroidNativeDragAndDrop.java | 55 +++++- Ports/iOSPort/nativeSources/CN1DragAndDrop.h | 21 ++- Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 159 +++++++++++++----- Ports/iOSPort/nativeSources/IOSNative.m | 20 ++- .../codename1/impl/ios/IOSImplementation.java | 29 +++- .../src/com/codename1/impl/ios/IOSNative.java | 27 +-- .../codename1/ui/NativeDragAndDropTest.java | 25 +++ 10 files changed, 305 insertions(+), 103 deletions(-) diff --git a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java index 2b350c09bc2..4f546a6d123 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java @@ -157,6 +157,7 @@ public static boolean startDrag(Component source, NativeDragOperation op) { return false; } op.setSource(source); + op.resetPerformedAction(); synchronized (LOCK) { active = op; currentTarget = null; @@ -204,6 +205,7 @@ public static NativeDragOperation dragSessionStarted() { currentTarget = null; currentAction = NativeDragOperation.ACTION_NONE; } + op.resetPerformedAction(); if (source != null) { // On the event dispatch thread, because it repaints. A component that is draggable // as well as a native drag source would otherwise be left mid-drag with its image diff --git a/CodenameOne/src/com/codename1/ui/NativeDragOperation.java b/CodenameOne/src/com/codename1/ui/NativeDragOperation.java index 49f43ddfd66..d4734443461 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragOperation.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragOperation.java @@ -250,6 +250,17 @@ public void removeCompletionListener(ActionListener l) { } } + /// Clears the outcome of a previous drag, because this operation is being installed as the + /// active one again. + /// + /// The same instance is offered for every drag of the component that owns it, so without + /// this `#getPerformedAction()` would go on reporting the *previous* drag's result for the + /// whole of the new one, which contradicts its contract that the value before completion is + /// `#ACTION_NONE`. + void resetPerformedAction() { + performedAction = ACTION_NONE; + } + /// Records the outcome and notifies the completion listeners. Invoked by the port, on the /// event dispatch thread, when the native drag session ends. /// diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java index 925f7902cb8..7ff7bdf39d4 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java @@ -10185,21 +10185,34 @@ public void run() { /// the clip, never null ClipData clipDataFor(ClipboardContent content) { int sdk = android.os.Build.VERSION.SDK_INT; - ClipData clip = null; - if (sdk >= 16 && content.getText(ClipboardContent.MIME_HTML) != null) { - clip = ClipData.newHtmlText("Codename One", - content.getText(ClipboardContent.MIME_TEXT), - content.getText(ClipboardContent.MIME_HTML)); - } else if (content.getText(ClipboardContent.MIME_TEXT) != null) { - clip = ClipData.newPlainText("Codename One", content.getText(ClipboardContent.MIME_TEXT)); + List mimeTypes = new ArrayList(); + List items = new ArrayList(); + String plain = content.getText(ClipboardContent.MIME_TEXT); + String html = content.getText(ClipboardContent.MIME_HTML); + if (sdk >= 16 && html != null) { + mimeTypes.add(ClipboardContent.MIME_TEXT); + mimeTypes.add(ClipboardContent.MIME_HTML); + items.add(new ClipData.Item(plain, html)); + } else if (plain != null) { + mimeTypes.add(ClipboardContent.MIME_TEXT); + items.add(new ClipData.Item(plain)); } try { - clip = enrichClipWithBinaryContent(content, clip); + addBinaryContent(content, mimeTypes, items); } catch (Throwable t) { com.codename1.io.Log.e(t); } - if (clip == null) { - clip = ClipData.newPlainText("Codename One", ""); + if (items.isEmpty()) { + return ClipData.newPlainText("Codename One", ""); + } + // Built from the union of the types, not by appending to a text clip. ClipData.addItem + // does not add the item's type to the description, so a clip assembled that way + // describes itself as text only -- and both a Codename One drop target filtering on + // MIME_FILE and an external receiver choosing a representation read the description. + ClipData clip = new ClipData("Codename One", + mimeTypes.toArray(new String[mimeTypes.size()]), items.get(0)); + for (int iter = 1; iter < items.size(); iter++) { + clip.addItem(items.get(iter)); } return clip; } @@ -10231,12 +10244,13 @@ public void cancelNativeDrag() { } /** - * Enriches the given base ClipData (which may be null) with image bytes and/or file - * references carried by the ClipboardContent, exposing binary content as FileProvider - * content:// URIs. Returns the (possibly newly created) ClipData, or the original clip on - * failure. Never throws. + * Collects the image bytes and file references carried by the ClipboardContent as items and + * MIME types, exposing binary content as FileProvider content:// URIs. The caller assembles + * the ClipData from the union of everything collected here and the text types, because + * ClipData.addItem cannot widen a description that already exists. */ - private ClipData enrichClipWithBinaryContent(ClipboardContent content, ClipData clip) throws IOException { + private void addBinaryContent(ClipboardContent content, List mimeTypes, + List items) throws IOException { String authority = getContext().getPackageName() + ".provider"; // Image bytes: prefer PNG, then JPEG, then GIF @@ -10272,11 +10286,10 @@ private ClipData enrichClipWithBinaryContent(ClipboardContent content, ClipData Uri imageUri = FileProvider.getUriForFile(getContext(), authority, imageFile); // Grant broadly so any paste target can read the content:// URI getContext().grantUriPermission("android", imageUri, Intent.FLAG_GRANT_READ_URI_PERMISSION); - if (clip == null) { - clip = new ClipData("Codename One", new String[]{ imageMime }, new ClipData.Item(imageUri)); - } else { - clip.addItem(new ClipData.Item(imageUri)); + if (!mimeTypes.contains(imageMime)) { + mimeTypes.add(imageMime); } + items.add(new ClipData.Item(imageUri)); } // File references: MIME_FILE may be a single String or a String[] @@ -10303,14 +10316,12 @@ private ClipData enrichClipWithBinaryContent(ClipboardContent content, ClipData u = FileProvider.getUriForFile(getContext(), authority, file); getContext().grantUriPermission("android", u, Intent.FLAG_GRANT_READ_URI_PERMISSION); } - if (clip == null) { - clip = new ClipData("Codename One", new String[]{ "text/uri-list" }, new ClipData.Item(u)); - } else { - clip.addItem(new ClipData.Item(u)); + if (!mimeTypes.contains("text/uri-list")) { + mimeTypes.add("text/uri-list"); } + items.add(new ClipData.Item(u)); } } - return clip; } /** diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java b/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java index a21195385f8..86b49dfcab6 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java @@ -66,6 +66,11 @@ final class AndroidNativeDragAndDrop { private static NativeDragOperation exporting; private static int lastAction = NativeDragOperation.ACTION_NONE; + /// What a drop of *our own* session onto one of our own components settled on, kept until + /// the session ends so the source is told what really happened. Android's drag events carry + /// no action, so this is the only place the true answer exists. + private static int localDropAction = NativeDragOperation.ACTION_NONE; + private static NativeDragOperation exporting() { synchronized (LOCK) { return exporting; @@ -90,6 +95,18 @@ private static void setLastAction(int action) { } } + private static int localDropAction() { + synchronized (LOCK) { + return localDropAction; + } + } + + private static void setLocalDropAction(int action) { + synchronized (LOCK) { + localDropAction = action; + } + } + private AndroidNativeDragAndDrop() { } @@ -145,6 +162,7 @@ static boolean startDrag(final AndroidImplementation impl, final NativeDragOpera } setExporting(op); setLastAction(NativeDragOperation.ACTION_NONE); + setLocalDropAction(NativeDragOperation.ACTION_NONE); view.post(new Runnable() { @Override public void run() { @@ -200,12 +218,15 @@ private static boolean handle(AndroidImplementation impl, DragEvent event) { return drop(impl, event); case DragEvent.ACTION_DRAG_ENDED: if (exporting() != null) { - int allowed = allowedActions(); + // Settled *before* the operation is forgotten. Reading the allowed + // actions afterwards is how this reported every move as a copy: with + // nothing exporting, allowedActions() answers with its copy fallback. + int completed = completedAction(event.getResult()); setExporting(null); - NativeDragAndDrop.dragCompleted(event.getResult() - ? preferred(allowed) : NativeDragOperation.ACTION_NONE); + NativeDragAndDrop.dragCompleted(completed); } setLastAction(NativeDragOperation.ACTION_NONE); + setLocalDropAction(NativeDragOperation.ACTION_NONE); return true; default: return false; @@ -216,6 +237,29 @@ private static boolean handle(AndroidImplementation impl, DragEvent event) { } } + /// What to tell the source a finished session actually did. + /// + /// A drop onto one of this application's own components knows exactly what was accepted, + /// and that is the answer -- without it a move accepted locally was reported as a copy and + /// a source relying on ACTION_MOVE to delete its data never did. + /// + /// A drop into *another* application cannot be answered so precisely: Android's drag + /// protocol carries no notion of copy versus move, and ACTION_DRAG_ENDED reports only a + /// boolean. Copy is the honest reading of "it succeeded and we do not know how", and it is + /// also the safe one, because reporting a move the receiver may not have performed would + /// have the source delete data nothing else holds. An operation that allows only a move + /// still reports a move, since there is nothing else it could have been. + private static int completedAction(boolean result) { + if (!result) { + return NativeDragOperation.ACTION_NONE; + } + int local = localDropAction(); + if (local != NativeDragOperation.ACTION_NONE) { + return local; + } + return preferred(allowedActions()); + } + private static boolean drop(AndroidImplementation impl, DragEvent event) { // A URI dropped by another application is only readable while this grant is held, and // the grant only exists from here on. Reading the content inside this method rather @@ -233,6 +277,11 @@ private static boolean drop(AndroidImplementation impl, DragEvent event) { ? preferred(allowedActions()) : lastAction(); int accepted = NativeDragAndDrop.drop(0, (int) event.getX(), (int) event.getY(), content, action); setLastAction(NativeDragOperation.ACTION_NONE); + if (exporting() != null) { + // Our own drag, dropped on our own surface: remember what the target took, because + // ACTION_DRAG_ENDED is about to be asked and has no way of knowing. + setLocalDropAction(accepted); + } return accepted != NativeDragOperation.ACTION_NONE; } diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.h b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h index 48da8d91918..69192ac5469 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.h +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h @@ -82,19 +82,26 @@ void CN1EnableNativeDropTarget(void); /// Stages the drag a press has made possible: which representations it can offer, what the /// receiver may do with them, and the image to show under the finger. The representations are -/// named but not built -- CN1SetNativeDragPayload delivers the bytes once the drag really +/// named but not built -- CN1AddNativeDragPayload delivers the bytes once the drag really /// starts. /// /// mimeTypes is newline separated. void CN1PrepareNativeDrag(NSString* mimeTypes, int allowedActions, NSData* dragImagePng, int touchX, int touchY); -/// Delivers the payload for the session UIKit has just started. Called from Java, from inside -/// the session-started callback below. +/// Clears the payload, ready for the representations of the session UIKit has just started. +/// Called from Java, from inside the session-started callback below. +void CN1BeginNativeDragPayload(void); + +/// Adds one representation to the payload being built. +/// +/// Every MIME type the operation advertises is pushed through here, rather than a fixed list of +/// the framework's own -- an operation carrying only, say, `text/markdown` was advertised but +/// never forwarded, so the drag began with no items and UIKit cancelled it at once. /// -/// fileUris is newline separated and may be nil. -void CN1SetNativeDragPayload(NSString* plain, NSString* html, NSString* rtf, - NSData* image, NSString* fileUris); +/// `text` and `binary` are alternatives; `application/x-file-list` arrives as newline separated +/// paths in `text`. +void CN1AddNativeDragPayload(NSString* mimeType, NSString* text, NSData* binary); /// Drops whatever CN1PrepareNativeDrag staged, because the press turned out to be a tap. void CN1CancelNativeDrag(void); @@ -116,7 +123,7 @@ int CN1NativeDragDeliverDrop(int x, int y, NSString* plain, NSString* html, NSSt /// Announces that UIKit has started a drag session. Returns the actions the framework's staged /// operation allows, or 0 when it has none -- in which case no drag begins. The Java side calls -/// CN1SetNativeDragPayload from inside this call. +/// CN1BeginNativeDragPayload and CN1AddNativeDragPayload from inside this call. int CN1NativeDragDeliverSessionStarted(void); /// Reports the outcome of a session this application started. diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index c791d1a9ee9..9b1c4462131 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -23,6 +23,12 @@ #import "CN1DragAndDrop.h" +#if !TARGET_OS_OSX && !TARGET_OS_WATCH && !TARGET_OS_TV +#if __has_include() +#import +#endif +#endif + #define CN1_DND_ACTION_NONE 0 #define CN1_DND_ACTION_COPY 1 #define CN1_DND_ACTION_MOVE 2 @@ -59,8 +65,10 @@ void CN1PrepareNativeDrag(NSString* mimeTypes, int allowedActions, NSData* dragI int touchX, int touchY) { } -void CN1SetNativeDragPayload(NSString* plain, NSString* html, NSString* rtf, - NSData* image, NSString* fileUris) { +void CN1BeginNativeDragPayload(void) { +} + +void CN1AddNativeDragPayload(NSString* mimeType, NSString* text, NSData* binary) { } void CN1CancelNativeDrag(void) { @@ -75,15 +83,20 @@ void CN1CancelNativeDrag(void) { /// What the press staged: the representations the drag could offer, what a receiver may do /// with them, and the preview. Named but not built -- see CN1DragAndDrop.h. static NSArray* cn1PreparedMimes = nil; -static int cn1PreparedActions = CN1_DND_ACTION_NONE; + +/// What a receiver is allowed to do with the drag. Staged by the press and replaced by the +/// authoritative set when a session really starts; it is what a *local* drop session is told +/// the source allows. UIKit carries no action information for a session that arrived from +/// another application, so those are told copy -- see cn1AllowedActionsFor. +static int cn1SessionActions = CN1_DND_ACTION_NONE; static UIImage* cn1PreparedPreview = nil; static CGPoint cn1PreparedTouch; /// The payload of the session UIKit is currently running, delivered by -/// CN1SetNativeDragPayload once the drag has actually begun. Keyed by uniform type identifier +/// CN1AddNativeDragPayload once the drag has actually begun. Keyed by uniform type identifier /// so the item provider can register each one directly. static NSMutableDictionary* cn1DragData = nil; -static NSArray* cn1DragFileUrls = nil; +static NSMutableArray* cn1DragFileUrls = nil; /// The last action the framework agreed to, reused when a drop arrives without one. static int cn1LastDropAction = CN1_DND_ACTION_NONE; @@ -127,7 +140,21 @@ void CN1CancelNativeDrag(void) { if ([mime isEqualToString:@"text/uri-list"]) { return @"public.url"; } - return nil; + // Anything else -- text/asciidoc, application/pdf, an application's own type -- still has + // to travel. Ask the system to name it, and fall back to the MIME type itself, which is an + // opaque identifier that a receiver which does not know it simply never asks for. + // Returning nil here would drop a representation the application deliberately published, + // and a drag whose *only* representation was one of those would begin with no items at all + // and be cancelled on the spot. +#if __has_include() + if (@available(iOS 14.0, *)) { + UTType* type = [UTType typeWithMIMEType:mime]; + if (type != nil && type.identifier.length > 0) { + return type.identifier; + } + } +#endif + return mime; } static NSString* cn1MimeForUti(NSString* uti) { @@ -159,19 +186,54 @@ void CN1CancelNativeDrag(void) { if ([uti isEqualToString:@"public.url"]) { return @"text/uri-list"; } + if ([uti containsString:@"/"]) { + // One of ours: cn1UtiForMime falls back to the MIME type as the identifier. + return uti; + } +#if __has_include() + if (@available(iOS 14.0, *)) { + UTType* type = [UTType typeWithIdentifier:uti]; + if (type != nil && type.preferredMIMEType.length > 0) { + return type.preferredMIMEType; + } + } +#endif + // A dynamic or private identifier with no MIME equivalent. Naming it anyway would fill the + // content with identifiers no drop target could match on. return nil; } -/// Files one representation under the uniform type identifier the rest of the system knows it -/// by, so the mapping lives in cn1UtiForMime rather than being spelled out again here. -static void cn1PutRepresentation(NSMutableDictionary* data, NSString* mime, NSData* value) { - if (value == nil || value.length == 0) { - return; +/// What the source of this drop session allows. +/// +/// A session this application started knows exactly, and using copy for it -- which is what +/// this did at first -- meant a move-only drag had no action in common with a move-only target +/// and could not be dropped at all, while a copy-or-move drag could only ever be proposed as a +/// copy, so no in-application reorder could report a move to its source. +/// +/// A session from another application is a different matter: UIKit tells a drop interaction +/// nothing about what the far side permits, so copy is the only defensible reading -- and the +/// safe one, since proposing a move the source never offered would have it delete data on the +/// strength of our guess. +static int cn1AllowedActionsFor(id session) { + if (session.localDragSession != nil && cn1SessionActions != CN1_DND_ACTION_NONE) { + return cn1SessionActions; + } + return CN1_DND_ACTION_COPY; +} + +/// One action out of a set, preferring a copy because it is the one that cannot destroy the +/// source's data. +static int cn1DefaultAction(int actions) { + if ((actions & CN1_DND_ACTION_COPY) != 0) { + return CN1_DND_ACTION_COPY; } - NSString* uti = cn1UtiForMime(mime); - if (uti != nil) { - [data setObject:value forKey:uti]; + if ((actions & CN1_DND_ACTION_MOVE) != 0) { + return CN1_DND_ACTION_MOVE; } + if ((actions & CN1_DND_ACTION_LINK) != 0) { + return CN1_DND_ACTION_LINK; + } + return CN1_DND_ACTION_NONE; } static UIDropOperation cn1DropOperationFor(int action) { @@ -242,7 +304,7 @@ void CN1PrepareNativeDrag(NSString* mimeTypes, int allowedActions, NSData* dragI [cn1PreparedPreview release]; #endif cn1PreparedMimes = mimes; - cn1PreparedActions = allowedActions; + cn1SessionActions = allowedActions; cn1PreparedPreview = dragImagePng == nil ? nil : [UIImage imageWithData:dragImagePng]; cn1PreparedTouch = CGPointMake(touchX / scaleValue, touchY / scaleValue); #ifndef CN1_USE_ARC @@ -251,16 +313,24 @@ void CN1PrepareNativeDrag(NSString* mimeTypes, int allowedActions, NSData* dragI #endif } -void CN1SetNativeDragPayload(NSString* plain, NSString* html, NSString* rtf, - NSData* image, NSString* fileUris) { - NSMutableDictionary* data = [NSMutableDictionary dictionary]; - cn1PutRepresentation(data, @"text/plain", plain == nil ? nil : [plain dataUsingEncoding:NSUTF8StringEncoding]); - cn1PutRepresentation(data, @"text/html", html == nil ? nil : [html dataUsingEncoding:NSUTF8StringEncoding]); - cn1PutRepresentation(data, @"text/rtf", rtf == nil ? nil : [rtf dataUsingEncoding:NSUTF8StringEncoding]); - cn1PutRepresentation(data, @"image/png", image); - NSMutableArray* urls = [NSMutableArray array]; - if (fileUris != nil && fileUris.length > 0) { - for (NSString* entry in [fileUris componentsSeparatedByString:@"\n"]) { +void CN1BeginNativeDragPayload(void) { +#ifndef CN1_USE_ARC + [cn1DragData release]; + [cn1DragFileUrls release]; +#endif + cn1DragData = [[NSMutableDictionary alloc] init]; + cn1DragFileUrls = [[NSMutableArray alloc] init]; +} + +void CN1AddNativeDragPayload(NSString* mimeType, NSString* text, NSData* binary) { + if (mimeType == nil || mimeType.length == 0 || cn1DragData == nil) { + return; + } + if ([mimeType isEqualToString:@"application/x-file-list"]) { + if (text == nil || text.length == 0) { + return; + } + for (NSString* entry in [text componentsSeparatedByString:@"\n"]) { if (entry.length == 0) { continue; } @@ -271,20 +341,22 @@ void CN1SetNativeDragPayload(NSString* plain, NSString* html, NSString* rtf, ? [NSURL fileURLWithPath:[entry stringByExpandingTildeInPath]] : [NSURL URLWithString:entry]; if (url != nil) { - [urls addObject:url]; + [cn1DragFileUrls addObject:url]; } } + return; + } + NSData* data = binary; + if (data == nil && text != nil) { + data = [text dataUsingEncoding:NSUTF8StringEncoding]; + } + if (data == nil || data.length == 0) { + return; + } + NSString* uti = cn1UtiForMime(mimeType); + if (uti != nil && [cn1DragData objectForKey:uti] == nil) { + [cn1DragData setObject:data forKey:uti]; } -#ifndef CN1_USE_ARC - [cn1DragData release]; - [cn1DragFileUrls release]; -#endif - cn1DragData = data; - cn1DragFileUrls = urls; -#ifndef CN1_USE_ARC - [cn1DragData retain]; - [cn1DragFileUrls retain]; -#endif } void CN1CancelNativeDrag(void) { @@ -294,7 +366,7 @@ void CN1CancelNativeDrag(void) { #endif cn1PreparedMimes = nil; cn1PreparedPreview = nil; - cn1PreparedActions = CN1_DND_ACTION_NONE; + cn1SessionActions = CN1_DND_ACTION_NONE; } API_AVAILABLE(ios(11.0)) @@ -312,11 +384,13 @@ @implementation CN1DragAndDropDelegate } // Asking the framework now, rather than on the press, is what lets a promised file stay // unwritten until a drag really happens. The Java side fills cn1DragData from inside this - // call through CN1SetNativeDragPayload. + // call, one representation at a time, through CN1AddNativeDragPayload. int allowed = CN1NativeDragDeliverSessionStarted(); if (allowed == CN1_DND_ACTION_NONE) { return @[]; } + // The authoritative set, which is what a local drop session is told the source allows. + cn1SessionActions = allowed; cn1DraggingOut = YES; NSMutableArray* items = [NSMutableArray array]; @@ -401,14 +475,16 @@ - (BOOL)dropInteraction:(UIDropInteraction *)interaction canHandleSession:(id)session { CGPoint point = [session locationInView:interaction.view]; cn1LastDropAction = CN1NativeDragDeliverOver((int)(point.x * scaleValue), (int)(point.y * scaleValue), - cn1MimesForSession(session), CN1_DND_ACTION_COPY, YES); + cn1MimesForSession(session), + cn1AllowedActionsFor(session), YES); } - (UIDropProposal *)dropInteraction:(UIDropInteraction *)interaction sessionDidUpdate:(id)session { CGPoint point = [session locationInView:interaction.view]; cn1LastDropAction = CN1NativeDragDeliverOver((int)(point.x * scaleValue), (int)(point.y * scaleValue), - cn1MimesForSession(session), CN1_DND_ACTION_COPY, NO); + cn1MimesForSession(session), + cn1AllowedActionsFor(session), NO); UIDropProposal* proposal = [[UIDropProposal alloc] initWithDropOperation:cn1DropOperationFor(cn1LastDropAction)]; #ifndef CN1_USE_ARC [proposal autorelease]; @@ -425,7 +501,8 @@ - (void)dropInteraction:(UIDropInteraction *)interaction performDrop:(id Date: Wed, 2 Sep 2026 09:11:31 +0300 Subject: [PATCH 04/26] Review: stop the bridges narrowing the payload on the way in as well The last round fixed this going out. The same defect was sitting on the receiving side of all three ports, and it has a sharper edge there: a drag is filtered twice -- once against what it advertises while it hovers, and again against what it materializes when it is dropped -- so a bridge that produces less than it advertised refuses the very target that just agreed to take it. **iOS dropped everything but five formats.** performDrop: loaded every registered type and then forwarded plain text, HTML, RTF, one image and files. A drag carrying markdown, a GIF or an application's own type was accepted while it hovered and arrived without it, so its target got no drop at all. Worse, the bridge's refusal was discarded: UIKit had already proposed an operation, so dragInteraction:session:didEndWithOperation: reported a move for a drop nothing received, and a source that deletes on ACTION_MOVE would delete data on the strength of it. The drop is now assembled one representation at a time, like the outbound payload, and the completion of a local drag waits for the drop's real answer -- UIKit asks the source what happened before the asynchronous loads have returned, so whichever arrives first now hands off to the other. **Android carried only text, images and files.** A content holding only MIME_MARKDOWN, MIME_ASCIIDOC or another byte-backed type produced an empty plain-text clip. A clip has one text payload, so where there is no text/plain the first text representation becomes that payload and its type is advertised with it; byte-backed types become typed content URIs, which is the only labelled way an Android clip carries bytes. A second, *different* text representation is deliberately not advertised: the clip cannot produce it, and advertising it is precisely how a target ends up accepting a hover it will then be refused. **Android lost the advertised types at materialization.** A URI item became MIME_FILE alone, so a component filtering on MIME_URI_LIST accepted the hover and was rejected at the drop. The drop now materializes with the description in hand and fills the types it advertised from what the clip actually produced -- nothing is invented, and a type with no value to give it is left absent rather than advertised empty. Paste passes no description and so is unchanged. **JavaSE discarded arbitrary binary flavors.** application/pdf and its like were refused on the way in purely for not being text or image, though readValue already handled streams and RichTransferable exports the same types on the way out. Any flavor in a shape this can read is now accepted, except AWT's own x-java transport flavors, which describe how a payload moves between Java processes rather than what it is. Co-Authored-By: Claude Opus 5 (1M context) --- .../impl/android/AndroidImplementation.java | 189 ++++++++++++++++-- .../android/AndroidNativeDragAndDrop.java | 5 +- .../impl/javase/JavaSENativeDragAndDrop.java | 22 ++ Ports/iOSPort/nativeSources/CN1DragAndDrop.h | 18 +- Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 75 +++++-- Ports/iOSPort/nativeSources/IOSNative.m | 25 ++- .../codename1/impl/ios/IOSImplementation.java | 49 +++-- 7 files changed, 321 insertions(+), 62 deletions(-) diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java index 7ff7bdf39d4..f1369a5d04a 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java @@ -10189,16 +10189,40 @@ ClipData clipDataFor(ClipboardContent content) { List items = new ArrayList(); String plain = content.getText(ClipboardContent.MIME_TEXT); String html = content.getText(ClipboardContent.MIME_HTML); + // A clip carries one text payload. Where the content has no text/plain but does have + // some other text representation -- markdown, AsciiDoc, a URI list -- that one is the + // payload, since publishing an empty clip instead would lose it outright. + String primaryTextMime = plain != null ? ClipboardContent.MIME_TEXT : null; + // Not when there is HTML: that is already the payload, and an item built with null text + // coerces to the HTML's own text, whereas handing it the raw markup as the plain text + // would give every receiver the tags as well. + if (plain == null && html == null) { + String[] advertised = content.getMimeTypes(); + for (int iter = 0; iter < advertised.length && plain == null; iter++) { + if (ClipboardContent.MIME_FILE.equals(advertised[iter])) { + continue; + } + String value = content.getText(advertised[iter]); + if (value != null) { + plain = value; + primaryTextMime = advertised[iter]; + } + } + } if (sdk >= 16 && html != null) { mimeTypes.add(ClipboardContent.MIME_TEXT); mimeTypes.add(ClipboardContent.MIME_HTML); items.add(new ClipData.Item(plain, html)); } else if (plain != null) { mimeTypes.add(ClipboardContent.MIME_TEXT); + if (primaryTextMime != null && !mimeTypes.contains(primaryTextMime)) { + mimeTypes.add(primaryTextMime); + } items.add(new ClipData.Item(plain)); } try { addBinaryContent(content, mimeTypes, items); + addRemainingRepresentations(content, plain, mimeTypes, items); } catch (Throwable t) { com.codename1.io.Log.e(t); } @@ -10271,25 +10295,13 @@ private void addBinaryContent(ClipboardContent content, List mimeTypes, imageExt = "gif"; } if (imageBytes != null) { - // AndroidGradleBuilder exposes cache/intent_files through the app's - // FileProvider. Keep generated clipboard payloads inside that root so - // FileProvider can safely create a content:// URI for paste targets. - File imageFile = new File(new File(getContext().getCacheDir(), "intent_files"), - "cn1-clip-image-" + System.currentTimeMillis() + "." + imageExt); - imageFile.getParentFile().mkdirs(); - OutputStream os = new FileOutputStream(imageFile); - try { - os.write(imageBytes); - } finally { - os.close(); - } - Uri imageUri = FileProvider.getUriForFile(getContext(), authority, imageFile); - // Grant broadly so any paste target can read the content:// URI - getContext().grantUriPermission("android", imageUri, Intent.FLAG_GRANT_READ_URI_PERMISSION); - if (!mimeTypes.contains(imageMime)) { - mimeTypes.add(imageMime); + Uri imageUri = writeAsProviderUri(imageBytes, imageExt); + if (imageUri != null) { + if (!mimeTypes.contains(imageMime)) { + mimeTypes.add(imageMime); + } + items.add(new ClipData.Item(imageUri)); } - items.add(new ClipData.Item(imageUri)); } // File references: MIME_FILE may be a single String or a String[] @@ -10324,6 +10336,83 @@ private void addBinaryContent(ClipboardContent content, List mimeTypes, } } + /// Adds the representations neither the text nor the binary pass above has taken. + /// + /// Byte-backed types -- a PDF, an archive, an application's own format -- become typed + /// content URIs, which is the only labelled way an Android clip carries bytes. Text types + /// are advertised only when their value *is* the text the clip already carries: a clip has + /// one text payload, so advertising a second, different reading of it would tell a receiver + /// the clip holds something it cannot then produce, and a Codename One target would accept + /// the hover and be refused at the drop. + private void addRemainingRepresentations(ClipboardContent content, String carriedText, + List mimeTypes, List items) throws IOException { + String[] advertised = content.getMimeTypes(); + for (int iter = 0; iter < advertised.length; iter++) { + String mime = advertised[iter]; + if (mimeTypes.contains(mime) || ClipboardContent.MIME_FILE.equals(mime)) { + continue; + } + Object value = content.getData(mime); + if (value instanceof String) { + if (carriedText != null && carriedText.equals(value)) { + mimeTypes.add(mime); + } + continue; + } + if (value instanceof byte[]) { + Uri uri = writeAsProviderUri((byte[]) value, extensionForMime(mime)); + if (uri != null) { + mimeTypes.add(mime); + items.add(new ClipData.Item(uri)); + } + } + } + } + + /// Writes bytes somewhere the application's file provider can serve them from and returns + /// the content URI, which is how an Android clip carries anything that is not text. + /// + /// AndroidGradleBuilder exposes cache/intent_files through the app's FileProvider, so + /// generated payloads stay inside that root and FileProvider can safely name them. + private Uri writeAsProviderUri(byte[] bytes, String extension) throws IOException { + if (bytes == null || bytes.length == 0) { + return null; + } + File file = new File(new File(getContext().getCacheDir(), "intent_files"), + "cn1-clip-" + System.currentTimeMillis() + "-" + bytes.length + "." + extension); + file.getParentFile().mkdirs(); + OutputStream os = new FileOutputStream(file); + try { + os.write(bytes); + } finally { + os.close(); + } + Uri uri = FileProvider.getUriForFile(getContext(), + getContext().getPackageName() + ".provider", file); + // Grant broadly so any paste or drop target can read the content:// URI + getContext().grantUriPermission("android", uri, Intent.FLAG_GRANT_READ_URI_PERMISSION); + return uri; + } + + /// A plausible file extension for a MIME type, used only to name the temporary file a + /// content URI is served from. + private static String extensionForMime(String mime) { + int slash = mime.indexOf('/'); + String sub = slash < 0 ? mime : mime.substring(slash + 1); + int plus = sub.indexOf('+'); + if (plus > 0) { + sub = sub.substring(0, plus); + } + StringBuilder out = new StringBuilder(); + for (int iter = 0; iter < sub.length(); iter++) { + char c = sub.charAt(iter); + if ((c >= 'a' && c <= 'z') || (c >= '0' && c <= '9')) { + out.append(c); + } + } + return out.length() == 0 ? "bin" : out.toString(); + } + /** * Maps a content resolver image MIME type to the corresponding ClipboardContent MIME constant, * defaulting to PNG for unrecognized image types. @@ -10391,6 +10480,31 @@ public void run() { /// /// the content, never null ClipboardContent contentFromClip(ClipData clip) { + return contentFromClip(clip, null); + } + + /// Reads a clip, and where a description is given also honours the MIME types it + /// advertises. + /// + /// A drag is filtered twice: once against the description while it hovers, and again + /// against the materialized content when it is dropped. If the second view is narrower than + /// the first, a target accepts the hover and is then refused the drop -- which is what + /// happened to a component filtering on `ClipboardContent#MIME_URI_LIST`, because a URI + /// item materializes as `MIME_FILE` alone. Nothing is invented here: an advertised type is + /// only filled from a value the clip actually produced. + /// + /// Paste passes no description, so it keeps reporting exactly what the clip contained. + /// + /// #### Parameters + /// + /// - `clip`: the clip data, which may be null + /// + /// - `description`: what the source advertised, or null to report only what was read + /// + /// #### Returns + /// + /// the content, never null + ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { ClipboardContent content = new ClipboardContent(); if (clip == null) { content.setData(ClipboardContent.MIME_TEXT, ""); @@ -10445,9 +10559,48 @@ ClipboardContent contentFromClip(ClipData clip) { content.setFiles(fileUris.toArray(new String[fileUris.size()])); } content.setData(ClipboardContent.MIME_TEXT, plain == null ? "" : plain); + if (description != null) { + fillAdvertisedTypes(content, description, plain, fileUris); + } return content; } + /// Fills the MIME types the drag advertised but the read did not produce, from what it did. + /// + /// An Android clip carries a single text payload and the description says what that text + /// is, so a type the description names and the clip did not otherwise yield is that text -- + /// `text/uri-list` excepted, which is the list of URIs the clip carried. A type with no + /// value to give it is left absent rather than advertised empty. + private static void fillAdvertisedTypes(ClipboardContent content, ClipDescription description, + String plain, List fileUris) { + for (int iter = 0; iter < description.getMimeTypeCount(); iter++) { + String mime = description.getMimeType(iter); + if (mime == null) { + continue; + } + mime = mime.toLowerCase(); + if (content.hasMimeType(mime)) { + continue; + } + if ("text/uri-list".equals(mime)) { + if (!fileUris.isEmpty()) { + StringBuilder uris = new StringBuilder(); + for (int j = 0; j < fileUris.size(); j++) { + if (j > 0) { + uris.append("\r\n"); + } + uris.append(fileUris.get(j)); + } + content.setData(ClipboardContent.MIME_URI_LIST, uris.toString()); + } + continue; + } + if (mime.startsWith("text/") && plain != null && plain.length() > 0) { + content.setData(mime, plain); + } + } + } + public static MediaException createMediaException(int extra) { MediaErrorType type; String message; diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java b/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java index 86b49dfcab6..5bf1aa3f5ea 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java @@ -272,7 +272,10 @@ private static boolean drop(AndroidImplementation impl, DragEvent event) { Log.e(err); } } - ClipboardContent content = impl.contentFromClip(event.getClipData()); + // With the description, so the types the drag advertised while it hovered survive into + // the content the drop is filtered against -- otherwise a target that accepted the + // hover on MIME_URI_LIST is refused the drop it was promised. + ClipboardContent content = impl.contentFromClip(event.getClipData(), event.getClipDescription()); int action = lastAction() == NativeDragOperation.ACTION_NONE ? preferred(allowedActions()) : lastAction(); int accepted = NativeDragAndDrop.drop(0, (int) event.getX(), (int) event.getY(), content, action); diff --git a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java index 23ba42c4a9e..1058fe4441c 100644 --- a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java +++ b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java @@ -285,9 +285,26 @@ private static String mimeFor(DataFlavor flavor) { if ("application/x-java-file-list".equals(mime)) { return ClipboardContent.MIME_FILE; } + if (mime.startsWith("application/x-java")) { + // AWT's own transport flavors -- serialized objects, local object references, the + // text-encoding list. They describe how a payload travels between Java processes, + // not what it is, and reading them can hand back arbitrary live objects. + return null; + } if (mime.startsWith("text/") || mime.startsWith("image/")) { return mime; } + // Anything else the source offers in a shape this can actually read. RichTransferable + // exports arbitrary binary types on the way out, so refusing them on the way in left a + // component filtered to, say, application/pdf unable to receive one at all. + Class representation = flavor.getRepresentationClass(); + if (representation != null + && (InputStream.class.isAssignableFrom(representation) + || byte[].class.equals(representation) + || String.class.equals(representation) + || java.io.Reader.class.isAssignableFrom(representation))) { + return mime; + } return null; } @@ -384,6 +401,11 @@ private static Object readValue(Transferable transferable, DataFlavor flavor, St if (out instanceof byte[]) { return out; } + if (!mime.startsWith("text/") && out instanceof InputStream) { + // A non-text type read as a stream is bytes, not characters; decoding it as + // UTF-8 the way the text path does would corrupt a PDF or an archive. + return readBytes((InputStream) out); + } return JavaSEPort.clipboardText(out); } catch (Throwable err) { return null; diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.h b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h index 69192ac5469..343516e16a6 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.h +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h @@ -117,9 +117,21 @@ int CN1NativeDragDeliverOver(int x, int y, NSString* mimeTypes, int allowedActio /// Reports a drag leaving the surface. void CN1NativeDragDeliverExit(void); -/// Delivers a drop and returns the action actually accepted, or 0 when nothing took it. -int CN1NativeDragDeliverDrop(int x, int y, NSString* plain, NSString* html, NSString* rtf, - NSData* image, NSString* fileUris, int action); +/// Starts a drop, clearing whatever a previous one left. +void CN1NativeDragDeliverDropBegin(void); + +/// Adds one representation to the drop being assembled. Every representation UIKit loaded goes +/// through here rather than a fixed list, so a drag carrying markdown, a GIF or an +/// application's own type is not accepted while it hovers and then found empty at the drop -- +/// which would refuse the very target that agreed to take it. +/// +/// `text` and `binary` are alternatives; `application/x-file-list` arrives as newline separated +/// paths in `text`. +void CN1NativeDragDeliverDropAdd(NSString* mimeType, NSString* text, NSData* binary); + +/// Delivers the assembled drop and returns the action actually accepted, or 0 when nothing took +/// it. +int CN1NativeDragDeliverDropCommit(int x, int y, int action); /// Announces that UIKit has started a drag session. Returns the actions the framework's staged /// operation allows, or 0 when it has none -- in which case no drag begins. The Java side calls diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index 9b1c4462131..def96075ca4 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -104,6 +104,20 @@ void CN1CancelNativeDrag(void) { /// True while this application is the source of the session in progress. static BOOL cn1DraggingOut = NO; +/// A drop of this application's own session onto its own surface, still loading. +/// +/// UIKit asks the source what happened -- dragInteraction:session:didEndWithOperation: -- as +/// soon as performDrop: returns, which is before the asynchronous loads that drop depends on +/// have answered. Reporting the operation UIKit proposed at that moment would tell a source +/// its data had been moved even when the framework went on to refuse the drop, and a source +/// that deletes on ACTION_MOVE would then delete data nothing ever received. So the completion +/// waits for the real answer, whichever of the two arrives first. +/// +/// All three are read and written on the main thread only, by callbacks UIKit serializes. +static BOOL cn1LocalDropInFlight = NO; +static BOOL cn1EndDeferred = NO; +static int cn1LocalDropResult = -1; + /// The surface, remembered so the drag interaction can be attached later, and the delegate that /// serves both interactions. The delegate outlives the surface, which lives for the life of the /// process, and the interactions hold it weakly. @@ -392,6 +406,9 @@ @implementation CN1DragAndDropDelegate // The authoritative set, which is what a local drop session is told the source allows. cn1SessionActions = allowed; cn1DraggingOut = YES; + cn1LocalDropInFlight = NO; + cn1EndDeferred = NO; + cn1LocalDropResult = -1; NSMutableArray* items = [NSMutableArray array]; // Files first, one item each: a receiver that copies documents expects one item per @@ -457,8 +474,17 @@ - (void)dragInteraction:(UIDragInteraction *)interaction session:(id)session didEndWithOperation:(UIDropOperation)operation { cn1DraggingOut = NO; + if (cn1LocalDropInFlight) { + // The drop landed here and is still assembling. Its answer is the true one, so the + // completion goes out when it arrives rather than on UIKit's proposal. + cn1EndDeferred = YES; + return; + } int action = CN1_DND_ACTION_NONE; - if (operation == UIDropOperationCopy) { + if (cn1LocalDropResult >= 0) { + action = cn1LocalDropResult; + cn1LocalDropResult = -1; + } else if (operation == UIDropOperationCopy) { action = CN1_DND_ACTION_COPY; } else if (operation == UIDropOperationMove) { action = CN1_DND_ACTION_MOVE; @@ -503,6 +529,11 @@ - (void)dropInteraction:(UIDropInteraction *)interaction performDrop:(id 0) { + CN1NativeDragDeliverDropAdd(@"application/x-file-list", + [files componentsJoinedByString:@"\n"], nil); } - NSString* plain = plainData == nil ? nil - : [[[NSString alloc] initWithData:plainData encoding:NSUTF8StringEncoding] autorelease]; - NSString* html = htmlData == nil ? nil - : [[[NSString alloc] initWithData:htmlData encoding:NSUTF8StringEncoding] autorelease]; - NSString* rtf = rtfData == nil ? nil - : [[[NSString alloc] initWithData:rtfData encoding:NSUTF8StringEncoding] autorelease]; - NSString* fileUris = files.count == 0 ? nil : [files componentsJoinedByString:@"\n"]; - CN1NativeDragDeliverDrop(x, y, plain, html, rtf, imageData, fileUris, action); + int accepted = CN1NativeDragDeliverDropCommit(x, y, action); cn1LastDropAction = CN1_DND_ACTION_NONE; + if (cn1LocalDropInFlight) { + cn1LocalDropInFlight = NO; + if (cn1EndDeferred) { + cn1EndDeferred = NO; + CN1NativeDragDeliverCompleted(accepted); + } else { + cn1LocalDropResult = accepted; + } + } #ifndef CN1_USE_ARC [collected release]; [files release]; diff --git a/Ports/iOSPort/nativeSources/IOSNative.m b/Ports/iOSPort/nativeSources/IOSNative.m index 5c8d2202959..76ba2155892 100644 --- a/Ports/iOSPort/nativeSources/IOSNative.m +++ b/Ports/iOSPort/nativeSources/IOSNative.m @@ -1120,16 +1120,21 @@ void CN1NativeDragDeliverExit(void) { com_codename1_impl_ios_IOSImplementation_nativeDragExitCallback__(CN1_THREAD_GET_STATE_PASS_SINGLE_ARG); } -int CN1NativeDragDeliverDrop(int x, int y, NSString* plain, NSString* html, NSString* rtf, - NSData* image, NSString* fileUris, int action) { - return (int)com_codename1_impl_ios_IOSImplementation_nativeDropCallback___int_int_java_lang_String_java_lang_String_java_lang_String_byte_1ARRAY_java_lang_String_int_R_int( - CN1_THREAD_GET_STATE_PASS_ARG x, y, - fromNSString(CN1_THREAD_GET_STATE_PASS_ARG plain), - fromNSString(CN1_THREAD_GET_STATE_PASS_ARG html), - fromNSString(CN1_THREAD_GET_STATE_PASS_ARG rtf), - image == nil ? JAVA_NULL : nsDataToByteArr(image), - fromNSString(CN1_THREAD_GET_STATE_PASS_ARG fileUris), - action); +void CN1NativeDragDeliverDropBegin(void) { + com_codename1_impl_ios_IOSImplementation_nativeDropBeginCallback__(CN1_THREAD_GET_STATE_PASS_SINGLE_ARG); +} + +void CN1NativeDragDeliverDropAdd(NSString* mimeType, NSString* text, NSData* binary) { + com_codename1_impl_ios_IOSImplementation_nativeDropAddCallback___java_lang_String_java_lang_String_byte_1ARRAY( + CN1_THREAD_GET_STATE_PASS_ARG + fromNSString(CN1_THREAD_GET_STATE_PASS_ARG mimeType), + fromNSString(CN1_THREAD_GET_STATE_PASS_ARG text), + binary == nil ? JAVA_NULL : nsDataToByteArr(binary)); +} + +int CN1NativeDragDeliverDropCommit(int x, int y, int action) { + return (int)com_codename1_impl_ios_IOSImplementation_nativeDropCommitCallback___int_int_int_R_int( + CN1_THREAD_GET_STATE_PASS_ARG x, y, action); } int CN1NativeDragDeliverSessionStarted(void) { diff --git a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java index 653b88aec8f..03a2f9ab563 100644 --- a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java +++ b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java @@ -9271,24 +9271,43 @@ public static void nativeDragExitCallback() { NativeDragAndDrop.dragExit(0); } - /// Invoked from CN1DragAndDrop.m with a fully loaded drop. Returns the action accepted, or - /// zero when nothing under the pointer took it. - public static int nativeDropCallback(int x, int y, String plain, String html, String rtf, - byte[] image, String fileUris, int action) { - ClipboardContent content = new ClipboardContent(); - if (plain != null) { - content.setData(ClipboardContent.MIME_TEXT, plain); - } - if (html != null) { - content.setData(ClipboardContent.MIME_HTML, html); + /// The drop being assembled by CN1DragAndDrop.m, one representation at a time. + /// + /// Only ever touched from the three callbacks below, which UIKit runs in order on the main + /// thread, so it needs no guarding of its own. + private static ClipboardContent pendingDrop; + + /// Invoked from CN1DragAndDrop.m as a drop begins, before its representations arrive. + public static void nativeDropBeginCallback() { + pendingDrop = new ClipboardContent(); + } + + /// Invoked from CN1DragAndDrop.m once per representation the drop carries. + /// + /// Every representation UIKit loaded is delivered here rather than a fixed list of the + /// framework's own: forwarding fewer meant a drag carrying markdown or an application's own + /// type was accepted while it hovered and then materialized without it, so the target that + /// agreed to take the drop was refused it. + public static void nativeDropAddCallback(String mimeType, String text, byte[] binary) { + if (pendingDrop == null || mimeType == null || mimeType.length() == 0) { + return; } - if (rtf != null) { - content.setData(ClipboardContent.MIME_RTF, rtf); + if (ClipboardContent.MIME_FILE.equals(mimeType)) { + pendingDrop.setFiles(split(text)); + return; } - if (image != null && image.length > 0) { - content.setData(ClipboardContent.MIME_PNG, image); + if (binary != null && binary.length > 0) { + pendingDrop.setData(mimeType, binary); + } else if (text != null && text.length() > 0) { + pendingDrop.setData(mimeType, text); } - content.setFiles(split(fileUris)); + } + + /// Invoked from CN1DragAndDrop.m once every representation has arrived. Returns the action + /// accepted, or zero when nothing under the pointer took it. + public static int nativeDropCommitCallback(int x, int y, int action) { + ClipboardContent content = pendingDrop == null ? new ClipboardContent() : pendingDrop; + pendingDrop = null; return NativeDragAndDrop.drop(0, x, y, content, action); } From e7a6b3b8433b805dc1d75578e7138316298397b0 Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 10:13:32 +0300 Subject: [PATCH 05/26] Review: one drag at a time, and a typed file is its type as well as a file Three more, all real, though one is fixed differently from the way it was put. **A second drag displaced the first.** startDrag installed the new operation before the port had answered, so a refusal cleared the session outright and a success attributed the running session's completion to the newcomer. Either way the original source never learned its outcome -- and one waiting for ACTION_MOVE to delete its data would wait for a completion that was no longer addressed to it. A start while a session is running is now refused, which is also all any of these platforms would have done. dragSessionStarted answers null in the same case, which is how a port whose platform owns the gesture declines. **A typed Android URI arrived as a file and nothing else.** A content: URI with type application/pdf became MIME_FILE alone, so a target filtering on the type accepted the hover -- the description advertised it -- and was refused the drop. The type is now offered as well, promised rather than read: a target that only wants the path should not pay for a document it never opens, and the drag-and-drop grant lasts the life of the activity, so the deferred read still succeeds. **An iOS file provider's other representations were skipped**, so the same advertise-then-refuse mismatch applied to a document dropped from Files. The review asked for the `continue` to be dropped, which would load every representation the provider offers -- and for a file provider that means reading the whole document into memory on top of the copy this already makes. A large video dropped from Files would be copied and then read into a byte array, which is a worse failure than the one being fixed. So the provider's other types are named against the copy instead and read only if a target asks for one: the advertised set and the deliverable set agree, which is the point, and nothing large is read that nobody wanted. That reasoning is in the code, since it is where the next reader will need it. The cast-semantics baseline is regenerated, and the diff is worth reading rather than trusting: two entries go because they are genuinely fixed -- AndroidDB's was corrected upstream by the portable-database change and never re-baselined, and the ClipboardContent cast is instanceof-guarded by this branch's own copyToClipboard refactor -- and the third moves from $62 to $63 because adding an anonymous class renumbered the ones after it. No finding is being silenced. Co-Authored-By: Claude Opus 5 (1M context) --- .../com/codename1/ui/NativeDragAndDrop.java | 20 ++++++++-- .../impl/android/AndroidImplementation.java | 38 ++++++++++++++++++- Ports/iOSPort/nativeSources/CN1DragAndDrop.h | 8 ++++ Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 31 ++++++++++++++- Ports/iOSPort/nativeSources/IOSNative.m | 7 ++++ .../codename1/impl/ios/IOSImplementation.java | 33 ++++++++++++++++ .../codename1/ui/NativeDragAndDropTest.java | 28 ++++++++++++++ scripts/cast-semantics-baseline.txt | 4 +- 8 files changed, 161 insertions(+), 8 deletions(-) diff --git a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java index 4f546a6d123..950e0dcc612 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java @@ -151,18 +151,27 @@ public static boolean isDragOutsideApplicationSupported() { /// #### Returns /// /// true when the operating system took the drag; false when the platform has no native drag - /// and drop, or refused to start a session + /// and drop, refused to start a session, or is already running one public static boolean startDrag(Component source, NativeDragOperation op) { if (op == null || !isSupported()) { return false; } - op.setSource(source); - op.resetPerformedAction(); synchronized (LOCK) { + if (active != null) { + // One drag at a time, which is all any of these platforms runs. Installing the + // second operation before the port has answered would strand the first: a + // refusal clears the session entirely and a success attributes the first + // session's completion to the second operation, so the original source never + // learns what happened -- and a source waiting for ACTION_MOVE to delete its + // data would wait forever. + return false; + } active = op; currentTarget = null; currentAction = NativeDragOperation.ACTION_NONE; } + op.setSource(source); + op.resetPerformedAction(); boolean started = false; try { started = Display.impl.startNativeDrag(op); @@ -194,6 +203,11 @@ public static NativeDragOperation dragSessionStarted() { NativeDragOperation op; final Component source; synchronized (LOCK) { + if (active != null) { + // A session is already running; see startDrag for why a second one must not + // displace it. The port refuses to start the drag on a null answer. + return null; + } op = pending; if (op == null) { return null; diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java index f1369a5d04a..a5704297cce 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java @@ -78,6 +78,7 @@ import com.codename1.ui.Image; import com.codename1.ui.PeerComponent; import com.codename1.ui.ClipboardContent; +import com.codename1.ui.ClipboardDataProvider; import com.codename1.ui.events.ActionEvent; import com.codename1.impl.CodenameOneImplementation; import com.codename1.impl.VirtualKeyboardInterface; @@ -10532,7 +10533,15 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { } continue; } - // Non-image URI -> file reference + // A typed URI is a file reference *and* that type. Reducing it to a file + // alone let a target filtering on, say, application/pdf accept the hover -- + // the description advertised the type -- and then be refused the drop, + // because the content it is filtered against a second time no longer had + // it. The bytes are promised rather than read: a target that only wants the + // path should not pay for a document it never opens. + if (type != null && type.length() > 0 && !content.hasMimeType(type)) { + content.setDataProvider(type.toLowerCase(), uriBytesProvider(uri)); + } fileUris.add(uri.toString()); continue; } @@ -10565,6 +10574,33 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { return content; } + /// Reads a content URI's bytes when something actually asks for them. + /// + /// The drag-and-drop permission this drop was granted lasts for the life of the activity -- + /// nothing calls release() on it -- so a read that happens a moment later on the event + /// dispatch thread still succeeds. + private ClipboardDataProvider uriBytesProvider(final Uri uri) { + return new ClipboardDataProvider() { + @Override + public Object getClipboardData(String mimeType) { + try { + InputStream in = getContext().getContentResolver().openInputStream(uri); + if (in == null) { + return null; + } + try { + return Util.readInputStream(in); + } finally { + in.close(); + } + } catch (Throwable t) { + com.codename1.io.Log.e(t); + return null; + } + } + }; + } + /// Fills the MIME types the drag advertised but the read did not produce, from what it did. /// /// An Android clip carries a single text payload and the description says what that text diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.h b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h index 343516e16a6..81a4dda6942 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.h +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h @@ -129,6 +129,14 @@ void CN1NativeDragDeliverDropBegin(void); /// paths in `text`. void CN1NativeDragDeliverDropAdd(NSString* mimeType, NSString* text, NSData* binary); +/// Adds a representation of the drop that is backed by a file already on disk. +/// +/// A provider that vends a file commonly advertises the document's own content type as well. +/// Loading that as data would read the whole document into memory on top of the copy this +/// already makes -- fatal for a large one -- so the type is named against the copy instead and +/// read only if a target asks for it. +void CN1NativeDragDeliverDropAddFile(NSString* mimeType, NSString* path); + /// Delivers the assembled drop and returns the action actually accepted, or 0 when nothing took /// it. int CN1NativeDragDeliverDropCommit(int x, int y, int action); diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index def96075ca4..ebd91558732 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -540,11 +540,21 @@ - (void)dropInteraction:(UIDropInteraction *)interaction performDrop:(id 0) { CN1NativeDragDeliverDropAdd(@"application/x-file-list", [files componentsJoinedByString:@"\n"], nil); @@ -626,6 +654,7 @@ - (void)dropInteraction:(UIDropInteraction *)interaction performDrop:(id Date: Wed, 2 Sep 2026 10:49:24 +0300 Subject: [PATCH 06/26] Review: a move nobody performed, a refusal that was overruled, a phone that grew up Three more, and the first is a correction to reasoning I wrote in the last round rather than to code I merely forgot to write. **Android reported a move nobody performed.** The completion for a successful external drop fell back to the source's preferred action, and I had defended that in a comment: an operation allowing only a move "still reports a move, since there is nothing else it could have been". That is wrong, and destructively so. What the source was willing to permit says nothing about what the receiver did -- Android's drag protocol has no notion of copy versus move at all, so an ordinary external target simply reads the clip. Reporting ACTION_MOVE on that basis has the documented completion handler delete the only remaining copy. A successful external drop now reports a copy whatever the source allowed, which is what actually happened. **Android overruled a target's refusal.** A target that calls NativeDropEvent.reject() leaves ACTION_NONE as the hover's answer, and Android delivers ACTION_DROP to a subscribed view regardless of what it answered to the location events. Treating that ACTION_NONE as "no answer yet" and substituting a default turned the refusal back into a delivered drop, against the contract that rejection prevents delivery. The last answer now distinguishes refused from not yet asked, and a refusal ends the drop and reports failure. iOS does not have the same hole and is deliberately left alone: UIKit consults the proposal from sessionDidUpdate: before it calls performDrop: at all, so a refusal means the drop never arrives and ACTION_NONE there really does mean "never updated". **iPhones can drag between applications now.** isDragOutsideApplicationSupported answered on the idiom alone, so every phone was told a drag could not leave the application. iOS 15 brought drag and drop between applications to the phone -- hold the item with one finger, switch applications with another, drop -- so an application hiding its export-by-drag affordance on this answer was hiding something the installed UIDragInteraction supports. Version gated now, and the developer guide's platform table says so rather than a flat no. Co-Authored-By: Claude Opus 5 (1M context) --- .../android/AndroidNativeDragAndDrop.java | 39 ++++++++++++++----- Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 19 ++++++--- .../Advanced-Topics-Under-The-Hood.asciidoc | 6 ++- 3 files changed, 47 insertions(+), 17 deletions(-) diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java b/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java index 5bf1aa3f5ea..b20c68c5502 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java @@ -64,7 +64,16 @@ final class AndroidNativeDragAndDrop { /// UI thread, so the lock is what publishes one to the other. private static final Object LOCK = new Object(); private static NativeDragOperation exporting; - private static int lastAction = NativeDragOperation.ACTION_NONE; + + /// What the framework last answered about a drag over this surface, or `UNDECIDED` before it + /// has answered anything. + /// + /// The distinction matters: `NativeDragOperation#ACTION_NONE` is a refusal, and Android + /// hands ACTION_DROP to a subscribed view whatever it answered to the location events -- so + /// treating a refusal as "no answer yet" and substituting a default turned a target's + /// reject() back into a delivered drop. + private static final int UNDECIDED = -1; + private static int lastAction = UNDECIDED; /// What a drop of *our own* session onto one of our own components settled on, kept until /// the session ends so the source is told what really happened. Android's drag events carry @@ -161,7 +170,7 @@ static boolean startDrag(final AndroidImplementation impl, final NativeDragOpera return false; } setExporting(op); - setLastAction(NativeDragOperation.ACTION_NONE); + setLastAction(UNDECIDED); setLocalDropAction(NativeDragOperation.ACTION_NONE); view.post(new Runnable() { @Override @@ -212,7 +221,7 @@ private static boolean handle(AndroidImplementation impl, DragEvent event) { return true; case DragEvent.ACTION_DRAG_EXITED: NativeDragAndDrop.dragExit(0); - setLastAction(NativeDragOperation.ACTION_NONE); + setLastAction(UNDECIDED); return true; case DragEvent.ACTION_DROP: return drop(impl, event); @@ -225,7 +234,7 @@ private static boolean handle(AndroidImplementation impl, DragEvent event) { setExporting(null); NativeDragAndDrop.dragCompleted(completed); } - setLastAction(NativeDragOperation.ACTION_NONE); + setLastAction(UNDECIDED); setLocalDropAction(NativeDragOperation.ACTION_NONE); return true; default: @@ -247,8 +256,9 @@ private static boolean handle(AndroidImplementation impl, DragEvent event) { /// protocol carries no notion of copy versus move, and ACTION_DRAG_ENDED reports only a /// boolean. Copy is the honest reading of "it succeeded and we do not know how", and it is /// also the safe one, because reporting a move the receiver may not have performed would - /// have the source delete data nothing else holds. An operation that allows only a move - /// still reports a move, since there is nothing else it could have been. + /// have the source delete data nothing else holds. That holds even for an operation that + /// allows only a move: what the source was willing to permit says nothing about what the + /// receiver did, and an ordinary Android target simply copies the clip. private static int completedAction(boolean result) { if (!result) { return NativeDragOperation.ACTION_NONE; @@ -257,7 +267,12 @@ private static int completedAction(boolean result) { if (local != NativeDragOperation.ACTION_NONE) { return local; } - return preferred(allowedActions()); + // A copy, whatever the source was willing to allow. An external receiver has read the + // clip and Android gives it no way to say more than that, so a copy is what actually + // happened. Answering ACTION_MOVE because the source offered nothing else would infer + // ownership from our own wishes: the receiver may simply have copied, and a source that + // deletes on ACTION_MOVE would then destroy the only remaining copy. + return NativeDragOperation.ACTION_COPY; } private static boolean drop(AndroidImplementation impl, DragEvent event) { @@ -276,8 +291,14 @@ private static boolean drop(AndroidImplementation impl, DragEvent event) { // the content the drop is filtered against -- otherwise a target that accepted the // hover on MIME_URI_LIST is refused the drop it was promised. ClipboardContent content = impl.contentFromClip(event.getClipData(), event.getClipDescription()); - int action = lastAction() == NativeDragOperation.ACTION_NONE - ? preferred(allowedActions()) : lastAction(); + int decided = lastAction(); + if (decided == NativeDragOperation.ACTION_NONE) { + // Refused while it hovered. Android delivers ACTION_DROP to a subscribed view even + // so, and substituting a default here is what turned a target's reject() back into + // a delivered drop. Reporting failure also makes ACTION_DRAG_ENDED report no action. + return false; + } + int action = decided == UNDECIDED ? preferred(allowedActions()) : decided; int accepted = NativeDragAndDrop.drop(0, (int) event.getX(), (int) event.getY(), content, action); setLastAction(NativeDragOperation.ACTION_NONE); if (exporting() != null) { diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index ebd91558732..6fba07a024d 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -295,18 +295,25 @@ BOOL CN1DragAndDropSupported(void) { } BOOL CN1DragOutsideAppSupported(void) { - if (@available(iOS 11.0, *)) { #if TARGET_OS_MACCATALYST + if (@available(iOS 11.0, *)) { return YES; + } + return NO; #else - // UIDragInteraction only carries a drag out of the application where the system has - // somewhere to carry it to: an iPad, or an iPhone running iPadOS style multitasking. - // On a phone in full screen the same session works, but it can only end on one of this - // application's own components. + // iOS 15 brought drag and drop between applications to the phone -- hold the item with one + // finger, switch applications with another, drop -- so from there on the idiom no longer + // decides it. Before that a drag could only leave the application on an iPad, where a + // second application can be on screen to receive it; on a phone the same session worked but + // could only end on one of this application's own components. + if (@available(iOS 15.0, *)) { + return YES; + } + if (@available(iOS 11.0, *)) { return [UIDevice currentDevice].userInterfaceIdiom == UIUserInterfaceIdiomPad; -#endif } return NO; +#endif } void CN1PrepareNativeDrag(NSString* mimeTypes, int allowedActions, NSData* dragImagePng, diff --git a/docs/developer-guide/Advanced-Topics-Under-The-Hood.asciidoc b/docs/developer-guide/Advanced-Topics-Under-The-Hood.asciidoc index 19febda4c1f..b1a90a8952a 100644 --- a/docs/developer-guide/Advanced-Topics-Under-The-Hood.asciidoc +++ b/docs/developer-guide/Advanced-Topics-Under-The-Hood.asciidoc @@ -1371,8 +1371,10 @@ travel as content URIs, the same way the clipboard carries them. |Drops into another application beside this one, and into Files or the Finder. |iPhone -|yes |no -|The session works, but a phone in full screen has no second application to drop into. +|yes |iOS 15 and later +|iOS 15 brought dragging between applications to the phone: hold the item with one finger, +switch applications with another, drop. Before that the session worked but could only end on one +of the application's own components. |Everything else |no |no From 144bb28e9321e9b4f15549f8b720fe963eec596c Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 11:24:46 +0300 Subject: [PATCH 07/26] Review: a disabled control could be dragged, and a stale callback could answer for a live drag **A disabled component staged a drag.** A Form primes drag and drop before it applies its own isEnabled gate, where a Window applies the gate first, so on the main surface a disabled native drag source staged an operation and the form level drag callback -- which runs before pressedCmp is consulted -- then started an operating system drag from a control that receives no ordinary press. The walk that looks for a drag source now skips components that are not enabled, so both surfaces behave alike, while an enabled draggable ancestor of a disabled child still drags exactly as the lightweight path lets it. **A stale callback could answer for a newer drag.** The callbacks are queued onto the event dispatch thread, so one can still be waiting when its drag leaves and another arrives over the same component. Guarding on component identity cannot tell those apart -- it is the same component -- so the old drag's decision was written into the new one's, and a move or a refusal from a drag that had already gone could be handed to a copy-only drag that had just arrived. Every target and session change now bumps a generation that each callback carries, and a callback only speaks while its own generation is current. The pending-dispatch flag is cleared on a target change for the same reason: its owner's callback will no longer clear it, and a flag left standing would silence the new target. The test for that one earned its keep the hard way. The obvious version passed with and without the fix: the corruption is repaired by the newer drag's own callback a moment later, so an assertion after the queue drains sees the right answer either way, and the recorder read its decision when it ran rather than when it was queued. It now decides from the payload and reads the answer from inside the queue, between the stale callback and the new drag's own, which is the only place the window is visible. Removing the guard makes it fail with the first drag's move where the second drag's copy belongs. Co-Authored-By: Claude Opus 5 (1M context) --- .../com/codename1/ui/NativeDragAndDrop.java | 36 +++++++- .../codename1/ui/NativeDragAndDropTest.java | 85 +++++++++++++++++++ 2 files changed, 118 insertions(+), 3 deletions(-) diff --git a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java index 950e0dcc612..42cdeec7d4a 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java @@ -113,6 +113,14 @@ public final class NativeDragAndDrop { private static int currentAction = NativeDragOperation.ACTION_NONE; private static boolean overDispatchPending; + /// Bumped whenever the target or the session changes. + /// + /// The callbacks below are queued onto the event dispatch thread, so one can still be + /// waiting when its drag leaves and another arrives over the same component. Identity alone + /// cannot tell those apart -- the component is the same one -- and the stale callback would + /// then answer for the new drag, handing it the old one's action or its refusal. + private static int targetGeneration; + private NativeDragAndDrop() { } @@ -168,6 +176,7 @@ public static boolean startDrag(Component source, NativeDragOperation op) { } active = op; currentTarget = null; + targetGeneration++; currentAction = NativeDragOperation.ACTION_NONE; } op.setSource(source); @@ -217,6 +226,7 @@ public static NativeDragOperation dragSessionStarted() { pendingSource = null; active = op; currentTarget = null; + targetGeneration++; currentAction = NativeDragOperation.ACTION_NONE; } op.resetPerformedAction(); @@ -265,7 +275,11 @@ static void pressedOn(Component cmp, int x, int y) { NativeDragOperation op = null; Component source = cmp; if (cmp != null && isSupported()) { - while (source != null && !source.isNativeDragSource()) { + // A disabled component is not a drag source, and neither is a disabled ancestor. + // A Form primes drag and drop before it applies its own isEnabled gate -- a Window + // applies it first -- so without this a disabled control could be dragged out of + // the application on the main surface and not in a window. + while (source != null && !(source.isNativeDragSource() && source.isEnabled())) { source = source.getParent(); } if (source != null) { @@ -504,6 +518,11 @@ public static int dragOver(int windowId, int x, int y, ClipboardContent content, changed = previous != target; // NOPMD CompareObjectsWithEquals if (changed) { currentTarget = target; + targetGeneration++; + // A pending dispatch belongs to the target that just went away, and its finally + // will no longer clear this -- so clearing it here is what keeps the new target + // able to dispatch at all. + overDispatchPending = false; currentAction = target == null ? NativeDragOperation.ACTION_NONE : preferredAction(allowedActions & target.getAcceptedDropActions()); } else if (target != null && !overDispatchPending) { @@ -531,6 +550,7 @@ public static void dragExit(int windowId) { synchronized (LOCK) { previous = currentTarget; currentTarget = null; + targetGeneration++; currentAction = NativeDragOperation.ACTION_NONE; } dispatch(previous, ActionEvent.Type.NativeDragExit, null, 0, 0, NativeDragOperation.ACTION_NONE); @@ -565,6 +585,7 @@ public static int drop(int windowId, int x, int y, ClipboardContent content, int : preferredAction(action & target.getAcceptedDropActions()); synchronized (LOCK) { currentTarget = null; + targetGeneration++; overDispatchPending = false; currentAction = accepted; } @@ -589,6 +610,7 @@ public static void dragCompleted(final int performedAction) { op = active; active = null; currentTarget = null; + targetGeneration++; currentAction = NativeDragOperation.ACTION_NONE; overDispatchPending = false; } @@ -681,9 +703,11 @@ private static void dispatch(final Component target, final ActionEvent.Type type } final boolean local; final int startingAction; + final int generation; synchronized (LOCK) { local = active != null; startingAction = currentAction; + generation = targetGeneration; } Display.getInstance().callSerially(new Runnable() { @Override @@ -699,7 +723,11 @@ public void run() { target.dispatchNativeDropEvent(ev); if (type == ActionEvent.Type.NativeDragOver || type == ActionEvent.Type.NativeDragEnter) { synchronized (LOCK) { - if (currentTarget == target) { // NOPMD CompareObjectsWithEquals + // The generation as well as the component: the same component can be + // the target of the drag that just left and of the one that just + // arrived, and this decision belongs to whichever queued it. + if (generation == targetGeneration + && currentTarget == target) { // NOPMD CompareObjectsWithEquals currentAction = ev.getAcceptedAction(); } } @@ -709,7 +737,9 @@ public void run() { } finally { if (type == ActionEvent.Type.NativeDragOver) { synchronized (LOCK) { - overDispatchPending = false; + if (generation == targetGeneration) { + overDispatchPending = false; + } } } } diff --git a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java index c09c3f242ca..834e6420182 100644 --- a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java +++ b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java @@ -453,6 +453,91 @@ void aDragSourceInsideADraggableContainerIsStillStaged() { } } + @FormTest + void aDisabledDragSourceIsNotDraggable() { + implementation.resetNativeDragState(); + implementation.setNativeDragAndDropSupported(true); + try { + Form form = Display.getInstance().getCurrent(); + Container source = new Container(); + source.setNativeDragOperation(new NativeDragOperation("dragged out")); + source.setEnabled(false); + form.setLayout(new BorderLayout()); + form.add(BorderLayout.CENTER, source); + form.revalidate(); + + int x = source.getAbsoluteX() + 10; + int y = source.getAbsoluteY() + 10; + form.pointerPressed(x, y); + assertNull(implementation.getPreparedNativeDrag(), + "a Form primes drag and drop before its own isEnabled gate, so the check " + + "has to be here or a disabled control is draggable on the main " + + "surface and not in a window"); + + form.pointerDragged(x + 200, y + 200); + assertNull(implementation.getStartedNativeDrag()); + form.pointerReleased(x + 200, y + 200); + + // Enabled again, the very same component drags. + source.setEnabled(true); + form.pointerPressed(x, y); + assertNotNull(implementation.getPreparedNativeDrag()); + form.pointerDragged(x + 200, y + 200); + assertNotNull(implementation.getStartedNativeDrag()); + NativeDragAndDrop.dragCompleted(NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + } finally { + implementation.setNativeDragAndDropSupported(false); + implementation.resetNativeDragState(); + } + } + + @FormTest + void aStaleDragCallbackDoesNotSpeakForANewerOne() { + Form form = Display.getInstance().getCurrent(); + // Decides from the payload rather than from a field, because the whole point is that + // the stale callback runs later and must still carry *its own* drag's decision. + Container target = new Container() { + @Override + protected void nativeDragEnter(NativeDropEvent ev) { + ev.accept("first".equals(ev.getText()) + ? NativeDragOperation.ACTION_MOVE : NativeDragOperation.ACTION_COPY); + } + }; + target.setNativeDropTarget(true); + form.setLayout(new BorderLayout()); + form.add(BorderLayout.CENTER, target); + form.revalidate(); + + int x = target.getAbsoluteX() + 5; + int y = target.getAbsoluteY() + 5; + // A drag arrives, settles on a move, and leaves -- with its callback still queued. + NativeDragAndDrop.dragEnter(0, x, y, textContent("first"), + NativeDragOperation.ACTION_COPY | NativeDragOperation.ACTION_MOVE); + NativeDragAndDrop.dragExit(0); + + // What the operating system would be told, read from between the stale callback and + // the new drag's own. Nothing else can see the window: once the new drag's callback + // runs it puts the right answer back, so an assertion after the queue drains would + // pass whether or not the stale one had spoken. + final int[] observed = { -1 }; + Display.getInstance().callSerially(new Runnable() { + public void run() { + observed[0] = NativeDragAndDrop.dragOver(0, x, y, textContent("second"), + NativeDragOperation.ACTION_COPY); + } + }); + + // The second, copy-only drag enters the same component before the queue drains. + NativeDragAndDrop.dragEnter(0, x, y, textContent("second"), NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + + assertEquals(NativeDragOperation.ACTION_COPY, observed[0], + "the first drag's move decision must not be handed to the second, copy-only drag"); + NativeDragAndDrop.dragExit(0); + flushSerialCalls(); + } + @FormTest void aClickOnANativeDragSourceDragsNothing() { implementation.resetNativeDragState(); From c48082d287c48a6ecfbf88b05b3c3e9e090f0137 Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 12:07:35 +0300 Subject: [PATCH 08/26] Review: the drop discarded the target's answer, and Android mislabelled what it carried **The shared drop path threw away the target's decision.** It recomputed the accepted action from what the port handed it and the target's declarative mask, and never looked at what the target's own callback had most recently said. The port's action is one drag event behind by construction -- it is whatever the last drag event returned -- so a target that called reject() in a callback that has since run had the refusal discarded and was handed the drop anyway. This is worth being clear about, because I reported it fixed two rounds ago. The Android port now refuses such a drop before the framework sees it, and that half is real: it makes Android report the drag as unsuccessful, which nothing else could. But it left JavaSE and iOS untouched, and I described the class of bug as closed. The drop now takes the target's latest word whenever the drop lands on the component the callbacks were about, and falls back to the declarative answer only when the pointer has moved to a different one. **Android dropped a distinct text representation rather than carrying it.** The previous round advertised a second text type only when its value matched the text the clip carries, on the grounds that advertising what cannot be produced is how a target accepts a hover and is then refused. That reasoning was sound and the conclusion was still wrong: a clip can carry the thing, as a typed content URI, exactly as binary travels. Markdown beside its plain rendering now goes out that way and comes back through the typed-URI provider, which decodes a text type to a String so getText() answers rather than returning bytes the caller cannot read. **Android filed WebP bytes as a PNG.** mimeForImageType answers PNG for any image type it does not recognize, so the bytes were stored under a label nothing could decode them by, and a target filtering on the type the drag advertised was accepted on the hover and refused at the drop. Incoming images now keep the type the content resolver reported. This is the same mislabelling as the JPEG published as PNG that the second round fixed on iOS; I did not think to look for Android's own version of it then. Both new tests were checked by removing the fix and watching them fail -- the rejection test reports a copy where none was allowed, and last round's stale callback test needed rewriting for exactly that reason. Co-Authored-By: Claude Opus 5 (1M context) --- .../com/codename1/ui/NativeDragAndDrop.java | 18 +++++++- .../impl/android/AndroidImplementation.java | 44 ++++++++++++++++--- .../codename1/ui/NativeDragAndDropTest.java | 24 ++++++++++ 3 files changed, 79 insertions(+), 7 deletions(-) diff --git a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java index 42cdeec7d4a..198ba3fdedc 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java @@ -581,9 +581,23 @@ public static void dragExit(int windowId) { /// the pointer took the drop and the port should report the transfer as failed public static int drop(int windowId, int x, int y, ClipboardContent content, int action) { Component target = findTarget(windowId, x, y, content); - int accepted = target == null ? NativeDragOperation.ACTION_NONE - : preferredAction(action & target.getAcceptedDropActions()); + int accepted; synchronized (LOCK) { + if (target != null && target == currentTarget) { // NOPMD CompareObjectsWithEquals + // The target's own latest word, not a recomputation from the action the port + // supplied. That action is by construction one event behind -- it is what the + // last drag event answered -- so a target that rejected, or changed its mind, + // in a callback that has since run would have had that decision quietly + // discarded here, and a refusal turned back into a delivered drop on every + // port rather than only the one that was noticed. + accepted = currentAction; + } else { + // A different component from the one the callbacks were about: the pointer + // moved between the last drag event and the drop, so there is no decision of + // its own to honour and the declarative answer is the right one. + accepted = target == null ? NativeDragOperation.ACTION_NONE + : preferredAction(action & target.getAcceptedDropActions()); + } currentTarget = null; targetGeneration++; overDispatchPending = false; diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java index a5704297cce..170db9b2cbc 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java @@ -10354,14 +10354,23 @@ private void addRemainingRepresentations(ClipboardContent content, String carrie continue; } Object value = content.getData(mime); + byte[] bytes = null; if (value instanceof String) { if (carriedText != null && carriedText.equals(value)) { + // The same text the clip already carries, so naming the type is enough. mimeTypes.add(mime); + continue; } - continue; + // A *different* reading -- Markdown source beside its plain rendering, say. + // A clip carries one text payload, so this one travels as a typed content URI + // the way binary does. Dropping it instead, which is what this did, lost a + // representation the application deliberately published. + bytes = ((String) value).getBytes("UTF-8"); + } else if (value instanceof byte[]) { + bytes = (byte[]) value; } - if (value instanceof byte[]) { - Uri uri = writeAsProviderUri((byte[]) value, extensionForMime(mime)); + if (bytes != null) { + Uri uri = writeAsProviderUri(bytes, extensionForMime(mime)); if (uri != null) { mimeTypes.add(mime); items.add(new ClipData.Item(uri)); @@ -10414,6 +10423,23 @@ private static String extensionForMime(String mime) { return out.length() == 0 ? "bin" : out.toString(); } + /// The MIME type to file an incoming image's bytes under: the framework's constant for the + /// three formats it names, and the type the content resolver reported for anything else. + /// + /// `#mimeForImageType(java.lang.String)` answers PNG for everything it does not recognize, + /// which for a WebP meant filing WebP bytes as a PNG -- undecodable by anything that + /// believed the label, and invisible to a target filtering on the type the drag advertised, + /// so the hover was accepted and the drop refused. + private static String imageMimeFor(String type) { + String lower = type.toLowerCase(); + if (lower.startsWith(ClipboardContent.MIME_PNG) + || lower.startsWith(ClipboardContent.MIME_JPEG) + || lower.startsWith(ClipboardContent.MIME_GIF)) { + return mimeForImageType(lower); + } + return lower; + } + /** * Maps a content resolver image MIME type to the corresponding ClipboardContent MIME constant, * defaulting to PNG for unrecognized image types. @@ -10526,7 +10552,7 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { if (in != null) { try { byte[] bytes = Util.readInputStream(in); - content.setData(mimeForImageType(type), bytes); + content.setData(imageMimeFor(type), bytes); } finally { in.close(); } @@ -10588,11 +10614,19 @@ public Object getClipboardData(String mimeType) { if (in == null) { return null; } + byte[] bytes; try { - return Util.readInputStream(in); + bytes = Util.readInputStream(in); } finally { in.close(); } + // A text type reads back as text: the framework's getText() answers null + // for a byte array, so a Markdown representation that went out as a typed + // URI would come back unreadable to the very API that asked for it. + if (bytes != null && mimeType != null && mimeType.startsWith("text/")) { + return new String(bytes, "UTF-8"); + } + return bytes; } catch (Throwable t) { com.codename1.io.Log.e(t); return null; diff --git a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java index 834e6420182..e028b3190e5 100644 --- a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java +++ b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java @@ -289,6 +289,30 @@ void dropDeliversTheContentAndNotifiesTheListener() { assertFalse(seen[0].isLocal(), "a drag this application did not start is not local"); } + @FormTest + void aRejectionMadeWhileHoveringSurvivesTheDrop() { + Form form = Display.getInstance().getCurrent(); + DropRecorder target = addTarget(form); + // The target refuses from inside its callback, which is the only place it can change + // its mind about a payload the declarative filters already let through. + target.rejectAction = NativeDragOperation.ACTION_NONE; + + int x = target.getAbsoluteX() + 5; + int y = target.getAbsoluteY() + 5; + NativeDragAndDrop.dragEnter(0, x, y, textContent("hi"), NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + + // The port drops carrying the action it was handed before that callback ran -- its + // answer is one drag event behind by construction -- so the refusal only survives if + // the drop honours the target's own latest word rather than recomputing. + assertEquals(NativeDragOperation.ACTION_NONE, + NativeDragAndDrop.drop(0, x, y, textContent("hi"), NativeDragOperation.ACTION_COPY), + "a target that refused while hovering must not be handed the drop anyway"); + flushSerialCalls(); + assertFalse(target.events.contains("drop"), + "and no drop event is delivered, which is what reject() promises"); + } + @FormTest void dropOnNothingReportsFailureSoThePortCanTellTheSource() { Form form = Display.getInstance().getCurrent(); From 574c3733f2438ab1231a6bb677be3d3cccfd03aa Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 13:18:26 +0300 Subject: [PATCH 09/26] Review: a leaked payload, a type identifier nobody recognised, a MIME lost in a URI **Every iOS drag leaked its whole payload.** Each representation was retained explicitly before its load handler was registered, but copying a block already retains what it captures -- and registering the handler copies it -- so the extra retain had nothing to balance it. Repeated drags of images or documents grew the footprint until the system took the application. Nothing local could have caught this: it compiles clean, passes every gate, and only shows on a device over many drags. **Below iOS 14 the type identifiers were meaningless.** UTType arrives in 14, so on 11 through 13 every type not named in the table -- application/pdf among them -- was published under its raw MIME string, which no application asking for com.adobe.pdf would ever match. That range is reachable: the builder defaults to 14 but ios.deployment_target lets an application go lower. Those releases now go through MobileCoreServices, with the deprecation silenced at the call rather than the call avoided, since it is the only way there to name a type the system knows. A dynamic identifier is refused, because it tells a receiver no more than the MIME type does and reads worse. **Android lost an application defined type inside its own URI.** The writers added in the previous rounds name the temporary file with an extension synthesized from the MIME type, and a FileProvider derives the URI's type from that extension -- so anything Android's table does not know came back as octet-stream and the advertised type was unrecoverable, leaving a target that accepted the hover refused at the drop. Android's own MimeTypeMap now supplies the extension wherever it has one, which settles every type it knows exactly. For the rest, a single unnamed URI is paired with a single unsatisfied advertised type, because that pairing cannot be anything else; with more of either it could be, so those are left absent and the target correctly refuses rather than being told it has something it may not. Co-Authored-By: Claude Opus 5 (1M context) --- .../impl/android/AndroidImplementation.java | 47 ++++++++++++++++--- Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 35 ++++++++++++-- 2 files changed, 71 insertions(+), 11 deletions(-) diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java index 170db9b2cbc..21017c68fde 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java @@ -10404,9 +10404,22 @@ private Uri writeAsProviderUri(byte[] bytes, String extension) throws IOExceptio return uri; } - /// A plausible file extension for a MIME type, used only to name the temporary file a - /// content URI is served from. + /// A file extension for a MIME type, used to name the temporary file a content URI is + /// served from. + /// + /// Android's own table first, because a FileProvider derives the URI's type from the + /// extension: a synthesized one it does not recognize makes ContentResolver.getType answer + /// application/octet-stream, and the type the clip advertised is then unrecoverable when + /// the clip is read back. private static String extensionForMime(String mime) { + try { + String known = android.webkit.MimeTypeMap.getSingleton().getExtensionFromMimeType(mime); + if (known != null && known.length() > 0) { + return known; + } + } catch (Throwable t) { + // Fall through to the synthesized extension below. + } int slash = mime.indexOf('/'); String sub = slash < 0 ? mime : mime.substring(slash + 1); int plus = sub.indexOf('+'); @@ -10541,6 +10554,9 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { String plain = null; String html = null; List fileUris = new ArrayList(); + // URIs the content resolver could not name. An application defined type has no entry in + // Android's table, so a FileProvider serving it reports octet-stream or nothing at all. + List unnamedUris = new ArrayList(); for (int i = 0; i < clip.getItemCount(); i++) { ClipData.Item item = clip.getItemAt(i); try { @@ -10565,8 +10581,13 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { // because the content it is filtered against a second time no longer had // it. The bytes are promised rather than read: a target that only wants the // path should not pay for a document it never opens. - if (type != null && type.length() > 0 && !content.hasMimeType(type)) { - content.setDataProvider(type.toLowerCase(), uriBytesProvider(uri)); + if (type != null && type.length() > 0 + && !"application/octet-stream".equals(type.toLowerCase())) { + if (!content.hasMimeType(type)) { + content.setDataProvider(type.toLowerCase(), uriBytesProvider(uri)); + } + } else { + unnamedUris.add(uri); } fileUris.add(uri.toString()); continue; @@ -10595,7 +10616,7 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { } content.setData(ClipboardContent.MIME_TEXT, plain == null ? "" : plain); if (description != null) { - fillAdvertisedTypes(content, description, plain, fileUris); + fillAdvertisedTypes(content, description, plain, fileUris, unnamedUris); } return content; } @@ -10641,8 +10662,9 @@ public Object getClipboardData(String mimeType) { /// is, so a type the description names and the clip did not otherwise yield is that text -- /// `text/uri-list` excepted, which is the list of URIs the clip carried. A type with no /// value to give it is left absent rather than advertised empty. - private static void fillAdvertisedTypes(ClipboardContent content, ClipDescription description, - String plain, List fileUris) { + private void fillAdvertisedTypes(ClipboardContent content, ClipDescription description, + String plain, List fileUris, List unnamedUris) { + List unsatisfiedBinary = new ArrayList(); for (int iter = 0; iter < description.getMimeTypeCount(); iter++) { String mime = description.getMimeType(iter); if (mime == null) { @@ -10652,6 +10674,9 @@ private static void fillAdvertisedTypes(ClipboardContent content, ClipDescriptio if (content.hasMimeType(mime)) { continue; } + if (!mime.startsWith("text/") && !"text/uri-list".equals(mime)) { + unsatisfiedBinary.add(mime); + } if ("text/uri-list".equals(mime)) { if (!fileUris.isEmpty()) { StringBuilder uris = new StringBuilder(); @@ -10669,6 +10694,14 @@ private static void fillAdvertisedTypes(ClipboardContent content, ClipDescriptio content.setData(mime, plain); } } + if (unsatisfiedBinary.size() == 1 && unnamedUris.size() == 1) { + // One type the clip promised and could not produce, and one URI whose type Android + // could not name: the pairing cannot be anything else. With more of either it could + // be, and inventing an association would tell a target it has something it may not + // -- which is the failure this whole path exists to avoid -- so those are left + // absent and the target correctly refuses. + content.setDataProvider(unsatisfiedBinary.get(0), uriBytesProvider(unnamedUris.get(0))); + } } public static MediaException createMediaException(int extra) { diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index 6fba07a024d..c63108c3bd1 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -24,6 +24,7 @@ #import "CN1DragAndDrop.h" #if !TARGET_OS_OSX && !TARGET_OS_WATCH && !TARGET_OS_TV +#import #if __has_include() #import #endif @@ -126,6 +127,26 @@ void CN1CancelNativeDrag(void) { /// The MIME types the framework names, mapped onto the uniform type identifiers UIKit and /// every other application on the system speak. +/// The uniform type identifier MobileCoreServices knows a MIME type by, or nil. +/// +/// Deprecated from iOS 15, which is why it is reached only when UTType is unavailable; the +/// warning is silenced rather than the call avoided, because on those releases it is the only +/// way to name a type the system will recognize. +static NSString* cn1LegacyUtiForMime(NSString* mime) { +#pragma clang diagnostic push +#pragma clang diagnostic ignored "-Wdeprecated-declarations" + CFStringRef identifier = UTTypeCreatePreferredIdentifierForTag(kUTTagClassMIMEType, + (CFStringRef) mime, NULL); +#pragma clang diagnostic pop + if (identifier == NULL) { + return nil; + } + NSString* result = [(NSString*) identifier autorelease]; + // A MIME type it does not know produces a dynamic identifier, which carries no more meaning + // to a receiver than the MIME type itself and reads far worse. + return [result hasPrefix:@"dyn."] ? nil : result; +} + static NSString* cn1UtiForMime(NSString* mime) { if ([mime isEqualToString:@"text/plain"]) { return @"public.utf8-plain-text"; @@ -166,9 +187,15 @@ void CN1CancelNativeDrag(void) { if (type != nil && type.identifier.length > 0) { return type.identifier; } + return mime; } #endif - return mime; + // Below iOS 14 UTType does not exist, and an application may deploy that far back through + // the ios.deployment_target build hint. Without this every type not named above -- including + // application/pdf -- was published under its MIME string, which no other application asking + // for the standard identifier would ever match. + NSString* legacy = cn1LegacyUtiForMime(mime); + return legacy != nil ? legacy : mime; } static NSString* cn1MimeForUti(NSString* uti) { @@ -436,9 +463,9 @@ @implementation CN1DragAndDropDelegate NSItemProvider* provider = [[NSItemProvider alloc] init]; for (NSString* uti in cn1DragData) { NSData* payload = [cn1DragData objectForKey:uti]; -#ifndef CN1_USE_ARC - [payload retain]; -#endif + // No retain of its own. Copying a block retains the objects it captures, which is + // what registering the load handler does, so an explicit retain here had nothing to + // balance it and every drag leaked its whole payload. [provider registerDataRepresentationForTypeIdentifier:uti visibility:NSItemProviderRepresentationVisibilityAll loadHandler:^NSProgress *(void (^completion)(NSData *, NSError *)) { From e28834d9698133de02a93a895616277a20de30aa Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 14:43:35 +0300 Subject: [PATCH 10/26] Review: the title area could not drag, and a promise was only a promise on one port **A Toolbar component could not be dragged.** Form.pointerPressed has a branch of its own for the title area, and it is the one branch that never primes drag and drop -- so a component given a native drag operation there silently could not be dragged while the identical component in the content pane could. Native dragging is primed there now. Only native: the lightweight drag has never worked in the title area either, and quietly switching that on is a different change from this one. **JavaSE committed an action the framework had not agreed to.** This is fallout from honouring the target's latest decision two commits ago. AWT wants the action when the drop is accepted, and that is before the transferable can be read, so accepting AWT's proposal and only then learning the target had chosen otherwise told the source through exportDone that a copy had happened while handing the target a move. NativeDragAndDrop.plannedDropAction answers the same question without dispatching anything, so what is committed to AWT is what the drop goes on to report. **iOS built every promised representation at the start of a drag.** Beginning a drag and abandoning it wrote every promised file and encoded every promised image, which is the opposite of what setDataProvider says. The item providers now resolve a representation when a receiver reads it, answering asynchronously so the fetch happens on the main thread like every other call into the framework from that file. The file list is the exception and stays eager: UIKit needs the number of items when the session begins, and for a file drag that number is the number of files -- deferring it would mean carrying only one, and dragging several files out is the feature. **Android cannot defer at all, so the promise was corrected instead of the code.** startDragAndDrop takes a complete ClipData, and a clip carries text or a URI to a file that already exists; there is no later moment to run a provider in. A content provider resolving bytes on demand would restore it and needs a second provider in the generated manifest, which lives in the builder repository, so it is not something this change can reach. ClipboardDataProvider, the Android bridge and the developer guide now each say where laziness holds and where it does not, and that a provider must be cheap enough to run once per drag. The javadoc promising more than two of the three ports could deliver was the actual defect. Also here: the casts in nativeDragResolveCallback moved out from under catch(Throwable). They were instanceof-guarded, which the cast-semantics gate does not recognize for an array type -- but the broad catch only ever needed to cover the provider call, which is the part that runs application code and can throw anything, so the narrower try is what should have been written. Co-Authored-By: Claude Opus 5 (1M context) --- .../codename1/ui/ClipboardDataProvider.java | 12 +++ CodenameOne/src/com/codename1/ui/Form.java | 6 ++ .../com/codename1/ui/NativeDragAndDrop.java | 38 ++++++++ .../android/AndroidNativeDragAndDrop.java | 13 +++ .../impl/javase/JavaSENativeDragAndDrop.java | 15 ++- Ports/iOSPort/nativeSources/CN1DragAndDrop.h | 23 +++-- Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 94 ++++++++++--------- Ports/iOSPort/nativeSources/IOSNative.m | 18 +++- .../codename1/impl/ios/IOSImplementation.java | 65 ++++++++++--- .../src/com/codename1/impl/ios/IOSNative.java | 19 ++-- .../Advanced-Topics-Under-The-Hood.asciidoc | 7 +- .../codename1/ui/NativeDragAndDropTest.java | 32 +++++++ 12 files changed, 260 insertions(+), 82 deletions(-) diff --git a/CodenameOne/src/com/codename1/ui/ClipboardDataProvider.java b/CodenameOne/src/com/codename1/ui/ClipboardDataProvider.java index dce7efccbc1..88774a78a7f 100644 --- a/CodenameOne/src/com/codename1/ui/ClipboardDataProvider.java +++ b/CodenameOne/src/com/codename1/ui/ClipboardDataProvider.java @@ -39,6 +39,18 @@ /// A provider is invoked at most once per `ClipboardContent` and MIME type -- the result is /// cached -- and it may be invoked from a native drag or clipboard thread rather than the event /// dispatch thread, so it must not touch the user interface. +/// +/// #### When it actually runs +/// +/// On the desktop and on iOS the provider runs when a receiver reads that representation, so a +/// drag the user abandons costs nothing. Android is the exception: `startDragAndDrop` takes a +/// complete clip, and a clip carries text or a reference to a file that already exists, so +/// every provider runs as the drag begins. A drag out of an iOS application resolves its *file +/// list* at the same moment for a related reason -- the system needs the number of items the +/// drag carries, and for a file drag that is the number of files. +/// +/// So a provider should be cheap enough to run once per drag, and must not assume it will only +/// run when its data is wanted. public interface ClipboardDataProvider { /// Produces the value for one representation. /// diff --git a/CodenameOne/src/com/codename1/ui/Form.java b/CodenameOne/src/com/codename1/ui/Form.java index 65a3b29cb92..db939ab24fa 100644 --- a/CodenameOne/src/com/codename1/ui/Form.java +++ b/CodenameOne/src/com/codename1/ui/Form.java @@ -4145,6 +4145,12 @@ public void pointerPressed(int x, int y) { if (cmp != null) { cmp = LeadUtil.leadParentImpl(cmp); + // Native drag and drop is primed here too. This branch does not call + // initDragAndDrop -- the lightweight drag has never worked in the title + // area -- so without this a Toolbar component given a native drag operation + // silently could not be dragged, while the same component in the content + // pane could. + NativeDragAndDrop.pressedOn(cmp, x, y); setPressedCmp(cmp); LeadUtil.pointerPressed(cmp, x, y); diff --git a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java index 198ba3fdedc..e72e185c393 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java @@ -610,6 +610,44 @@ public static int drop(int windowId, int x, int y, ClipboardContent content, int return accepted; } + /// The action a drop at this position would perform, without dispatching anything or + /// disturbing the drag in progress. + /// + /// A port whose platform commits to an action *before* it can read the transferred data -- + /// AWT does, because a drop has to be accepted before it becomes readable -- asks here + /// first, so that what it commits to is what `#drop(int, int, int, + /// com.codename1.ui.ClipboardContent, int)` will go on to report. Committing the platform's + /// own stale action instead told the source a copy had happened while the target was handed + /// a move. + /// + /// #### Parameters + /// + /// - `windowId`: the id of the window the drag is over, or zero for the main surface + /// + /// - `x`: the pointer position within that surface + /// + /// - `y`: the pointer position within that surface + /// + /// - `content`: the representations the drag is offering, which may still be a description + /// rather than the materialized payload + /// + /// - `action`: the action the platform is proposing + /// + /// #### Returns + /// + /// the action the drop would perform, or `NativeDragOperation#ACTION_NONE` + public static int plannedDropAction(int windowId, int x, int y, ClipboardContent content, + int action) { + Component target = findTarget(windowId, x, y, content); + synchronized (LOCK) { + if (target != null && target == currentTarget) { // NOPMD CompareObjectsWithEquals + return currentAction; + } + return target == null ? NativeDragOperation.ACTION_NONE + : preferredAction(action & target.getAcceptedDropActions()); + } + } + /// Reports that the session started by `#startDrag(com.codename1.ui.Component, /// com.codename1.ui.NativeDragOperation)` has finished, whatever the outcome, so that a /// source offering `NativeDragOperation#ACTION_MOVE` learns whether to delete its copy. diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java b/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java index b20c68c5502..001cb43e226 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java @@ -376,6 +376,19 @@ public Object getClipboardData(String requested) { /// `ClipboardContent` into a clip -- including writing image bytes out through the /// application's file provider so another application can read them -- so this reuses that /// rather than growing a second conversion that would drift from it. + /// + /// #### Promised representations are produced here, not on demand + /// + /// `ClipboardContent#setDataProvider(java.lang.String, com.codename1.ui.ClipboardDataProvider)` + /// is honoured lazily on the desktop and on iOS, where the platform asks for a + /// representation when a receiver reads it. Android has no such moment: startDragAndDrop + /// takes a complete `android.content.ClipData`, and a clip carries text or a URI to a file + /// that already exists. So every provider runs when the drag begins, including for a drag + /// the user goes on to abandon. + /// + /// A content provider resolving its bytes on demand would restore the guarantee, but it + /// needs a second provider declared in the manifest -- which is generated by the builder, + /// in another repository -- so it is not something this file can reach for. private static ClipData toClipData(AndroidImplementation impl, ClipboardContent content) { try { return impl.clipDataFor(content); diff --git a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java index 1058fe4441c..e732264e8e3 100644 --- a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java +++ b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java @@ -541,6 +541,16 @@ public void drop(DropTargetDropEvent e) { if (action == NativeDragOperation.ACTION_NONE) { action = preferred(allowed); } + Point at = e.getLocation(); + int x = canvas.scaleCoordinateX(at.x); + int y = canvas.scaleCoordinateY(at.y); + // What the framework will settle on, asked before anything is committed. AWT + // has to be told the action when the drop is accepted, and that is before the + // transferable can be read -- so accepting the action AWT proposed and only + // then learning the target had chosen another reported a copy to the source + // through exportDone while handing the target a move. + action = NativeDragAndDrop.plannedDropAction(canvas.windowId, x, y, + contentFor(e.getTransferable(), e.getCurrentDataFlavors(), false), action); if (action == NativeDragOperation.ACTION_NONE) { e.rejectDrop(); return; @@ -551,10 +561,7 @@ public void drop(DropTargetDropEvent e) { // than handed to the event dispatch thread as a live view of the transfer. e.acceptDrop(toAwtAction(action)); ClipboardContent content = contentFor(e.getTransferable(), e.getCurrentDataFlavors(), true); - Point at = e.getLocation(); - int accepted = NativeDragAndDrop.drop(canvas.windowId, - canvas.scaleCoordinateX(at.x), canvas.scaleCoordinateY(at.y), - content, action); + int accepted = NativeDragAndDrop.drop(canvas.windowId, x, y, content, action); e.dropComplete(accepted != NativeDragOperation.ACTION_NONE); } catch (Throwable err) { Log.e(err); diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.h b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h index 81a4dda6942..9120e37268c 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.h +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h @@ -93,15 +93,20 @@ void CN1PrepareNativeDrag(NSString* mimeTypes, int allowedActions, NSData* dragI /// Called from Java, from inside the session-started callback below. void CN1BeginNativeDragPayload(void); -/// Adds one representation to the payload being built. +/// Names a representation the drag can offer without producing it. /// -/// Every MIME type the operation advertises is pushed through here, rather than a fixed list of -/// the framework's own -- an operation carrying only, say, `text/markdown` was advertised but -/// never forwarded, so the drag began with no items and UIKit cancelled it at once. +/// The bytes are fetched through CN1NativeDragDeliverResolve when a receiver actually reads +/// that type, which is what setDataProvider promises: beginning a drag and abandoning it must +/// not build anything. Every MIME type the operation advertises is declared here, rather than a +/// fixed list of the framework's own -- an operation carrying only `text/markdown` was +/// advertised and never forwarded, so the drag began with no items and UIKit cancelled it. +void CN1DeclareNativeDragPayload(NSString* mimeType); + +/// Adds the file list, which is the one representation that cannot be deferred: UIKit needs the +/// number of items when the session begins, and that is the number of files. /// -/// `text` and `binary` are alternatives; `application/x-file-list` arrives as newline separated -/// paths in `text`. -void CN1AddNativeDragPayload(NSString* mimeType, NSString* text, NSData* binary); +/// `paths` is newline separated. +void CN1AddNativeDragFiles(NSString* paths); /// Drops whatever CN1PrepareNativeDrag staged, because the press turned out to be a tap. void CN1CancelNativeDrag(void); @@ -141,6 +146,10 @@ void CN1NativeDragDeliverDropAddFile(NSString* mimeType, NSString* path); /// it. int CN1NativeDragDeliverDropCommit(int x, int y, int action); +/// Produces one representation of the drag in progress, on demand. Returns nil when the +/// operation cannot supply it. +NSData* CN1NativeDragDeliverResolve(NSString* mimeType); + /// Announces that UIKit has started a drag session. Returns the actions the framework's staged /// operation allows, or 0 when it has none -- in which case no drag begins. The Java side calls /// CN1BeginNativeDragPayload and CN1AddNativeDragPayload from inside this call. diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index c63108c3bd1..97d35d3d102 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -69,7 +69,10 @@ void CN1PrepareNativeDrag(NSString* mimeTypes, int allowedActions, NSData* dragI void CN1BeginNativeDragPayload(void) { } -void CN1AddNativeDragPayload(NSString* mimeType, NSString* text, NSData* binary) { +void CN1DeclareNativeDragPayload(NSString* mimeType) { +} + +void CN1AddNativeDragFiles(NSString* paths) { } void CN1CancelNativeDrag(void) { @@ -94,9 +97,9 @@ void CN1CancelNativeDrag(void) { static CGPoint cn1PreparedTouch; /// The payload of the session UIKit is currently running, delivered by -/// CN1AddNativeDragPayload once the drag has actually begun. Keyed by uniform type identifier -/// so the item provider can register each one directly. -static NSMutableDictionary* cn1DragData = nil; +/// CN1DeclareNativeDragPayload once the drag has actually begun -- the MIME types it can offer, +/// not their values, which are fetched only if a receiver reads them. +static NSMutableArray* cn1DragMimes = nil; static NSMutableArray* cn1DragFileUrls = nil; /// The last action the framework agreed to, reused when a drop arrives without one. @@ -363,47 +366,38 @@ void CN1PrepareNativeDrag(NSString* mimeTypes, int allowedActions, NSData* dragI void CN1BeginNativeDragPayload(void) { #ifndef CN1_USE_ARC - [cn1DragData release]; + [cn1DragMimes release]; [cn1DragFileUrls release]; #endif - cn1DragData = [[NSMutableDictionary alloc] init]; + cn1DragMimes = [[NSMutableArray alloc] init]; cn1DragFileUrls = [[NSMutableArray alloc] init]; } -void CN1AddNativeDragPayload(NSString* mimeType, NSString* text, NSData* binary) { - if (mimeType == nil || mimeType.length == 0 || cn1DragData == nil) { - return; - } - if ([mimeType isEqualToString:@"application/x-file-list"]) { - if (text == nil || text.length == 0) { - return; - } - for (NSString* entry in [text componentsSeparatedByString:@"\n"]) { - if (entry.length == 0) { - continue; - } - // ClipboardContent's file representation permits a raw local path as well as a - // file: URI, and URLWithString: turns a path into a scheme-less relative URL that - // no receiver can open. - NSURL* url = ([entry hasPrefix:@"/"] || [entry hasPrefix:@"~"]) - ? [NSURL fileURLWithPath:[entry stringByExpandingTildeInPath]] - : [NSURL URLWithString:entry]; - if (url != nil) { - [cn1DragFileUrls addObject:url]; - } - } +void CN1DeclareNativeDragPayload(NSString* mimeType) { + if (mimeType == nil || mimeType.length == 0 || cn1DragMimes == nil + || [cn1DragMimes containsObject:mimeType]) { return; } - NSData* data = binary; - if (data == nil && text != nil) { - data = [text dataUsingEncoding:NSUTF8StringEncoding]; - } - if (data == nil || data.length == 0) { + [cn1DragMimes addObject:mimeType]; +} + +void CN1AddNativeDragFiles(NSString* paths) { + if (paths == nil || paths.length == 0 || cn1DragFileUrls == nil) { return; } - NSString* uti = cn1UtiForMime(mimeType); - if (uti != nil && [cn1DragData objectForKey:uti] == nil) { - [cn1DragData setObject:data forKey:uti]; + for (NSString* entry in [paths componentsSeparatedByString:@"\n"]) { + if (entry.length == 0) { + continue; + } + // ClipboardContent's file representation permits a raw local path as well as a file: + // URI, and URLWithString: turns a path into a scheme-less relative URL that no receiver + // can open. + NSURL* url = ([entry hasPrefix:@"/"] || [entry hasPrefix:@"~"]) + ? [NSURL fileURLWithPath:[entry stringByExpandingTildeInPath]] + : [NSURL URLWithString:entry]; + if (url != nil) { + [cn1DragFileUrls addObject:url]; + } } } @@ -431,8 +425,9 @@ @implementation CN1DragAndDropDelegate return @[]; } // Asking the framework now, rather than on the press, is what lets a promised file stay - // unwritten until a drag really happens. The Java side fills cn1DragData from inside this - // call, one representation at a time, through CN1AddNativeDragPayload. + // unwritten until a drag really happens. The Java side names its representations from + // inside this call, through CN1DeclareNativeDragPayload; their values are fetched only if a + // receiver reads them. int allowed = CN1NativeDragDeliverSessionStarted(); if (allowed == CN1_DND_ACTION_NONE) { return @[]; @@ -459,17 +454,26 @@ @implementation CN1DragAndDropDelegate [item release]; #endif } - if (cn1DragData.count > 0) { + if (cn1DragMimes.count > 0) { NSItemProvider* provider = [[NSItemProvider alloc] init]; - for (NSString* uti in cn1DragData) { - NSData* payload = [cn1DragData objectForKey:uti]; - // No retain of its own. Copying a block retains the objects it captures, which is - // what registering the load handler does, so an explicit retain here had nothing to - // balance it and every drag leaked its whole payload. + for (NSString* mime in cn1DragMimes) { + NSString* uti = cn1UtiForMime(mime); + if (uti == nil) { + continue; + } + // The value is fetched when a receiver reads this type, not now: a drag that is + // begun and abandoned must not have written the file or encoded the image it was + // merely offering. NSItemProvider allows a load handler to answer asynchronously, + // which is what lets the fetch happen on the main thread where every other call + // into the framework from this file happens. No retain of the captured strings + // either -- copying a block retains what it captures, and an explicit retain here + // leaked the whole payload of every drag. [provider registerDataRepresentationForTypeIdentifier:uti visibility:NSItemProviderRepresentationVisibilityAll loadHandler:^NSProgress *(void (^completion)(NSData *, NSError *)) { - completion(payload, nil); + dispatch_async(dispatch_get_main_queue(), ^{ + completion(CN1NativeDragDeliverResolve(mime), nil); + }); return nil; }]; } diff --git a/Ports/iOSPort/nativeSources/IOSNative.m b/Ports/iOSPort/nativeSources/IOSNative.m index ba7a417bd04..b0236fb70c2 100644 --- a/Ports/iOSPort/nativeSources/IOSNative.m +++ b/Ports/iOSPort/nativeSources/IOSNative.m @@ -1085,13 +1085,17 @@ void com_codename1_impl_ios_IOSNative_beginNativeDragPayload__(CN1_THREAD_STATE_ CN1BeginNativeDragPayload(); } -void com_codename1_impl_ios_IOSNative_addNativeDragPayload___java_lang_String_java_lang_String_byte_1ARRAY(CN1_THREAD_STATE_MULTI_ARG JAVA_OBJECT instanceObject, JAVA_OBJECT mimeType, JAVA_OBJECT text, JAVA_OBJECT binary) { +void com_codename1_impl_ios_IOSNative_declareNativeDragPayload___java_lang_String(CN1_THREAD_STATE_MULTI_ARG JAVA_OBJECT instanceObject, JAVA_OBJECT mimeType) { POOL_BEGIN(); // Synchronously, unlike prepare: these run inside the session-started callback on the main // thread and the item providers are built from them the moment it returns. - CN1AddNativeDragPayload(mimeType == JAVA_NULL ? nil : toNSString(CN1_THREAD_STATE_PASS_ARG mimeType), - text == JAVA_NULL ? nil : toNSString(CN1_THREAD_STATE_PASS_ARG text), - binary == JAVA_NULL ? nil : arrayToData(binary)); + CN1DeclareNativeDragPayload(mimeType == JAVA_NULL ? nil : toNSString(CN1_THREAD_STATE_PASS_ARG mimeType)); + POOL_END(); +} + +void com_codename1_impl_ios_IOSNative_addNativeDragFiles___java_lang_String(CN1_THREAD_STATE_MULTI_ARG JAVA_OBJECT instanceObject, JAVA_OBJECT paths) { + POOL_BEGIN(); + CN1AddNativeDragFiles(paths == JAVA_NULL ? nil : toNSString(CN1_THREAD_STATE_PASS_ARG paths)); POOL_END(); } @@ -1139,6 +1143,12 @@ void CN1NativeDragDeliverDropAddFile(NSString* mimeType, NSString* path) { fromNSString(CN1_THREAD_GET_STATE_PASS_ARG path)); } +NSData* CN1NativeDragDeliverResolve(NSString* mimeType) { + JAVA_OBJECT bytes = com_codename1_impl_ios_IOSImplementation_nativeDragResolveCallback___java_lang_String_R_byte_1ARRAY( + CN1_THREAD_GET_STATE_PASS_ARG fromNSString(CN1_THREAD_GET_STATE_PASS_ARG mimeType)); + return bytes == JAVA_NULL ? nil : arrayToData(bytes); +} + int CN1NativeDragDeliverDropCommit(int x, int y, int action) { return (int)com_codename1_impl_ios_IOSImplementation_nativeDropCommitCallback___int_int_int_R_int( CN1_THREAD_GET_STATE_PASS_ARG x, y, action); diff --git a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java index 9f7167e041b..36552425be0 100644 --- a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java +++ b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java @@ -9355,31 +9355,66 @@ public static int nativeDragSessionStartedCallback() { } ClipboardContent content = op.getContent(); nativeInstance.beginNativeDragPayload(); - // Every advertised representation, not a fixed list of the framework's own. prepare - // advertises whatever the content holds, so forwarding less than that left an operation - // carrying only, say, MIME_MARKDOWN advertising a type it then could not produce -- and - // a drag with no items at all is cancelled the moment it starts. + // Names, not values. Reading a representation here would build every promised file and + // encode every promised image at the moment the drag begins -- including for a drag the + // user then abandons -- which is the opposite of what setDataProvider promises. The + // native side asks for one through nativeDragResolveCallback if a receiver reads it. String[] mimeTypes = content.getMimeTypes(); for (int iter = 0; iter < mimeTypes.length; iter++) { String mime = mimeTypes[iter]; if (ClipboardContent.MIME_FILE.equals(mime)) { - // Normalized through getFiles(), which reads back both the single-String and - // the String[] spellings a producer may have used. - nativeInstance.addNativeDragPayload(mime, join(content.getFiles()), null); + // The exception, and it is UIKit's: the session needs its item count when it + // begins, and for a file drag that count is the number of files. So this one + // representation is resolved now. + nativeInstance.addNativeDragFiles(join(content.getFiles())); continue; } - // Reading the value here is what resolves a promised representation, which is the - // whole point of doing this at session start rather than on the press. - Object value = content.getData(mime); - if (value instanceof String) { - nativeInstance.addNativeDragPayload(mime, (String) value, null); - } else if (value instanceof byte[]) { - nativeInstance.addNativeDragPayload(mime, null, (byte[]) value); - } + nativeInstance.declareNativeDragPayload(mime); } return op.getAllowedActions(); } + /// Invoked from CN1DragAndDrop.m when a receiver reads one of the drag's representations. + /// + /// This is where a promised value is finally produced -- the file written, the image + /// encoded -- so a drag that nobody reads costs nothing. + /// + /// #### Parameters + /// + /// - `mimeType`: the representation being read + /// + /// #### Returns + /// + /// its bytes, or null when the operation cannot supply it + public static byte[] nativeDragResolveCallback(String mimeType) { + NativeDragOperation op = NativeDragAndDrop.getActiveDrag(); + if (op == null || mimeType == null || mimeType.length() == 0) { + return null; + } + // Only the provider call is inside the broad catch: it runs application code, which may + // throw anything. The casts below sit outside it deliberately -- a cast reached through + // catch(Throwable) is exactly what ParparVM cannot report, since its CHECKCAST does not + // throw and the handler would never run on iOS. + Object value; + try { + value = op.getContent().getData(mimeType); + } catch (Throwable err) { + com.codename1.io.Log.e(err); + return null; + } + if (value instanceof byte[]) { + return (byte[]) value; + } + if (value instanceof String) { + try { + return ((String) value).getBytes("UTF-8"); + } catch (java.io.UnsupportedEncodingException err) { + com.codename1.io.Log.e(err); + } + } + return null; + } + /// Invoked from CN1DragAndDrop.m when a session this application started has ended. public static void nativeDragCompletedCallback(int action) { NativeDragAndDrop.dragCompleted(action); diff --git a/Ports/iOSPort/src/com/codename1/impl/ios/IOSNative.java b/Ports/iOSPort/src/com/codename1/impl/ios/IOSNative.java index 71c93dee38b..090025daff4 100644 --- a/Ports/iOSPort/src/com/codename1/impl/ios/IOSNative.java +++ b/Ports/iOSPort/src/com/codename1/impl/ios/IOSNative.java @@ -380,21 +380,28 @@ native void prepareNativeDrag(String mimeTypes, int allowedActions, byte[] dragI /// built only once a drag really happens. native void beginNativeDragPayload(); - /// Adds one representation to the payload being built. + /// Names a representation the drag can offer, without producing it. /// /// Every MIME type the operation advertises goes through here rather than a fixed list, so /// nothing the application published is silently left behind -- a drag offering only - /// `text/markdown` used to advertise it and then carry nothing, which UIKit cancels. + /// `text/markdown` used to advertise it and then carry nothing, which UIKit cancels. The + /// value is produced only if a receiver reads that type, which is what + /// `com.codename1.ui.ClipboardContent#setDataProvider(java.lang.String, com.codename1.ui.ClipboardDataProvider)` + /// promises. /// /// #### Parameters /// /// - `mimeType`: the representation's MIME type + native void declareNativeDragPayload(String mimeType); + + /// Adds the file list, the one representation that cannot be deferred: UIKit needs the + /// number of items when the session begins, and for a file drag that is the number of + /// files. /// - /// - `text`: the value when it is text, or newline separated paths for - /// `com.codename1.ui.ClipboardContent#MIME_FILE`; null otherwise + /// #### Parameters /// - /// - `binary`: the value when it is binary; null otherwise - native void addNativeDragPayload(String mimeType, String text, byte[] binary); + /// - `paths`: newline separated file paths or `file:` URIs + native void addNativeDragFiles(String paths); /// Drops whatever `#prepareNativeDrag(java.lang.String, int, byte[], int, int)` staged, /// because the press turned out to be a tap. diff --git a/docs/developer-guide/Advanced-Topics-Under-The-Hood.asciidoc b/docs/developer-guide/Advanced-Topics-Under-The-Hood.asciidoc index b1a90a8952a..a183ccd80a0 100644 --- a/docs/developer-guide/Advanced-Topics-Under-The-Hood.asciidoc +++ b/docs/developer-guide/Advanced-Topics-Under-The-Hood.asciidoc @@ -1313,13 +1313,18 @@ include::../demos/common/src/main/java/com/codenameone/developerguide/advancedto Dragging a file works the same way, except that the file often doesn't exist yet and the user may drop it nowhere at all. Register it as a provider and it's written at the moment a receiver -asks for it, and never otherwise: +asks for it rather than when the drag starts: [source,java] ---- include::../demos/common/src/main/java/com/codenameone/developerguide/advancedtopics/NativeDragAndDropDemo.java[tag=nativeFileDrag,indent=0] ---- +Two platforms can't defer quite that far. An Android clip has to be complete before the drag +begins, so a provider runs there as the drag starts; an iOS drag resolves its file list at the +same moment, because the system needs to know how many items the drag carries. Keep a provider +cheap enough to run once per drag. + `NativeDragOperation.ACTION_MOVE` means the receiver takes ownership and the source deletes its copy. The source only learns whether that happened once the operating system has finished, which is why the outcome arrives through the completion listener rather than from the call that started diff --git a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java index e028b3190e5..88c401c1c64 100644 --- a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java +++ b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java @@ -477,6 +477,38 @@ void aDragSourceInsideADraggableContainerIsStillStaged() { } } + @FormTest + void aDragSourceInTheTitleAreaIsStaged() { + implementation.resetNativeDragState(); + implementation.setNativeDragAndDropSupported(true); + try { + Form form = Display.getInstance().getCurrent(); + // The title area has its own press branch, and it is the one branch that never + // primed drag and drop -- so a Toolbar component given an operation could not be + // dragged while the same component in the content pane could. + Container source = new Container(); + source.setNativeDragOperation(new NativeDragOperation("from the title")); + source.setPreferredSize(new com.codename1.ui.geom.Dimension(40, 40)); + form.getTitleArea().add(BorderLayout.EAST, source); + form.revalidate(); + + int x = source.getAbsoluteX() + 2; + int y = source.getAbsoluteY() + 2; + assertTrue(y < form.getContentPane().getAbsoluteY(), + "the fixture has to sit in the title area for this to test anything"); + + form.pointerPressed(x, y); + assertNotNull(implementation.getPreparedNativeDrag()); + form.pointerDragged(x + 200, y + 200); + assertNotNull(implementation.getStartedNativeDrag()); + NativeDragAndDrop.dragCompleted(NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + } finally { + implementation.setNativeDragAndDropSupported(false); + implementation.resetNativeDragState(); + } + } + @FormTest void aDisabledDragSourceIsNotDraggable() { implementation.resetNativeDragState(); From f5447378f869472c59e136c61cbee93d1b419bd3 Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 15:10:33 +0300 Subject: [PATCH 11/26] Review: the lifting preview was the wrong class, and the desktop modifier did nothing **iOS was handed an object of a class it did not ask for.** UIDragInteractionDelegate declares previewForLiftingItem: as returning a UITargetedDragPreview; this returned a UIDragPreview, which is an unrelated class, so UIKit was going to send it messages it does not answer. Clang says nothing about the mismatch -- the file compiles without a single warning -- and only a drag on a device with a custom drag image would have found it. The review found it from the other end: cn1PreparedTouch was being written and never read, so setDragImageOffset had no effect. It has none because an untargeted preview is positioned wherever UIKit likes; the fix is the targeted preview the delegate was asking for all along, placed so the point the finger grabbed stays under the finger. Every other delegate method in the file was checked against the SDK headers rather than only this one -- the other seven match. Compile-clean with no warnings at iOS 11, 14 and 15; the iOS 11 spellings UIDragPreviewTarget and UIDragPreviewParameters are used deliberately, because UITargetedPreview and UIPreviewTarget arrive in 13 and this feature claims 11. **The desktop modifier could not select a move.** getSourceActions is the whole mask the source offered, and handing the framework that alone made it prefer a copy every time -- so holding the platform modifier changed nothing, because getDropAction, which is where AWT records the user's choice, was never read. That choice now wins where the source allows it, and the full mask stands where it does not. Co-Authored-By: Claude Opus 5 (1M context) --- .../impl/javase/JavaSENativeDragAndDrop.java | 15 +++++++++- Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 28 +++++++++++++++---- 2 files changed, 37 insertions(+), 6 deletions(-) diff --git a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java index e732264e8e3..c92be2370a7 100644 --- a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java +++ b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java @@ -573,13 +573,26 @@ public void drop(DropTargetDropEvent e) { } } + /// What the source permits *at this instant*. + /// + /// getSourceActions is the whole mask the source offered, and answering with it alone + /// made the framework prefer a copy every time -- so holding the platform's modifier to + /// ask for a move changed nothing, because getDropAction, which is where AWT records + /// that choice, was never read. The user's choice wins where the source allows it, and + /// the full mask stands where it does not. + private int allowedActionsFor(DropTargetDragEvent e) { + int sourceActions = fromAwtActions(e.getSourceActions()); + int chosen = fromAwtActions(e.getDropAction()) & sourceActions; + return chosen == NativeDragOperation.ACTION_NONE ? sourceActions : chosen; + } + private void respond(DropTargetDragEvent e, boolean entering) { try { Point at = e.getLocation(); int x = canvas.scaleCoordinateX(at.x); int y = canvas.scaleCoordinateY(at.y); ClipboardContent content = contentFor(e.getTransferable(), e.getCurrentDataFlavors(), false); - int allowed = fromAwtActions(e.getSourceActions()); + int allowed = allowedActionsFor(e); int action = entering ? NativeDragAndDrop.dragEnter(canvas.windowId, x, y, content, allowed) : NativeDragAndDrop.dragOver(canvas.windowId, x, y, content, allowed); diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index 97d35d3d102..78979156a7b 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -491,18 +491,36 @@ @implementation CN1DragAndDropDelegate return items; } -- (UIDragPreview *)dragInteraction:(UIDragInteraction *)interaction - previewForLiftingItem:(UIDragItem *)item - session:(id)session { - if (cn1PreparedPreview == nil) { +- (UITargetedDragPreview *)dragInteraction:(UIDragInteraction *)interaction + previewForLiftingItem:(UIDragItem *)item + session:(id)session { + // UITargetedDragPreview, which is what UIDragInteractionDelegate declares. This returned a + // UIDragPreview instead -- an unrelated class -- so UIKit was handed an object it would go + // on to send UITargetedDragPreview messages to. Clang does not warn about the mismatch, and + // nothing but a device would have shown it. + if (cn1PreparedPreview == nil || interaction.view == nil) { // Without one UIKit snapshots the interaction's view, which is the whole Codename One // surface; nil here leaves UIKit to its default rather than dragging the entire screen. return nil; } UIImageView* view = [[UIImageView alloc] initWithImage:cn1PreparedPreview]; - UIDragPreview* preview = [[UIDragPreview alloc] initWithView:view]; + // Positioned so the point the finger grabbed stays under the finger. Untargeted, the + // preview is centred whereever UIKit chooses and the image jumps out from under the touch + // the moment the drag lifts -- which is also why setDragImageOffset had no effect at all. + CGPoint touch = [session locationInView:interaction.view]; + CGSize size = cn1PreparedPreview.size; + CGPoint centre = CGPointMake(touch.x - cn1PreparedTouch.x + size.width / 2.0, + touch.y - cn1PreparedTouch.y + size.height / 2.0); + UIDragPreviewTarget* target = [[UIDragPreviewTarget alloc] initWithContainer:interaction.view + center:centre]; + UIDragPreviewParameters* parameters = [[UIDragPreviewParameters alloc] init]; + UITargetedDragPreview* preview = [[UITargetedDragPreview alloc] initWithView:view + parameters:parameters + target:target]; #ifndef CN1_USE_ARC [view release]; + [target release]; + [parameters release]; [preview autorelease]; #endif return preview; From 05423cceb747876e93c68cbb2d883e3d2d28c602 Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 15:39:27 +0300 Subject: [PATCH 12/26] Review: one direction of a mapping, an empty payload, and text returned as bytes Three of my own, and the first is the shape worth naming: a fix applied to one direction of a symmetric pair and not the other. **The legacy type mapping only went one way.** Two rounds ago the MIME to UTI conversion below iOS 14 was fixed through MobileCoreServices, and the reverse -- UTI to MIME -- was left answering nil unconditionally on those releases. So on iOS 11 through 13 a standard type such as com.adobe.pdf was still neither discovered while a drag hovered nor materialized when it dropped. The diff looked complete because the direction it touched was complete. **Empty was being treated as absent.** A drop representation had to have a positive length to be stored, so a representation the drag advertised and that is legitimately empty -- an empty string, a zero byte payload -- vanished, and the drop was then refused by the very target the hover had accepted. Null is absent; empty is present. Android's provider writer had the identical test and is fixed with it rather than waiting to be found separately. **File-backed text came back as bytes.** A document provider from Files offers a plain text representation beside its file URL, and the file-backed provider always answered with a byte array, so getText() and NativeDropEvent.getText() were null for a type the drop had just accepted. It decodes text/* as UTF-8 now, which is what the Android provider and the other iOS drop path already did -- so this was an inconsistency between three paths that should have read alike. Co-Authored-By: Claude Opus 5 (1M context) --- .../impl/android/AndroidImplementation.java | 5 +++- Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 23 +++++++++++++++++++ .../codename1/impl/ios/IOSImplementation.java | 19 ++++++++++++--- 3 files changed, 43 insertions(+), 4 deletions(-) diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java index 21017c68fde..d501f8721df 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java @@ -10385,9 +10385,12 @@ private void addRemainingRepresentations(ClipboardContent content, String carrie /// AndroidGradleBuilder exposes cache/intent_files through the app's FileProvider, so /// generated payloads stay inside that root and FileProvider can safely name them. private Uri writeAsProviderUri(byte[] bytes, String extension) throws IOException { - if (bytes == null || bytes.length == 0) { + if (bytes == null) { return null; } + // A zero length payload is still a payload: refusing it would leave the clip without a + // type it had advertised, and a target filtering on that type would accept the hover + // and be refused the drop. File file = new File(new File(getContext().getCacheDir(), "intent_files"), "cn1-clip-" + System.currentTimeMillis() + "-" + bytes.length + "." + extension); file.getParentFile().mkdirs(); diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index 78979156a7b..9eaa76cdd06 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -201,6 +201,20 @@ void CN1CancelNativeDrag(void) { return legacy != nil ? legacy : mime; } +/// The MIME type MobileCoreServices knows a uniform type identifier by, or nil. The reverse of +/// cn1LegacyUtiForMime, and deprecated from iOS 15 for the same reason. +static NSString* cn1LegacyMimeForUti(NSString* uti) { +#pragma clang diagnostic push +#pragma clang diagnostic ignored "-Wdeprecated-declarations" + CFStringRef mime = UTTypeCopyPreferredTagWithClass((CFStringRef) uti, kUTTagClassMIMEType); +#pragma clang diagnostic pop + if (mime == NULL) { + return nil; + } + NSString* result = [(NSString*) mime autorelease]; + return result.length > 0 ? [result lowercaseString] : nil; +} + static NSString* cn1MimeForUti(NSString* uti) { if ([uti isEqualToString:@"public.utf8-plain-text"] || [uti isEqualToString:@"public.plain-text"] || [uti isEqualToString:@"public.text"]) { @@ -240,8 +254,17 @@ void CN1CancelNativeDrag(void) { if (type != nil && type.preferredMIMEType.length > 0) { return type.preferredMIMEType; } + return nil; } #endif + // Below iOS 14, the same way round as cn1UtiForMime goes. Answering nil here regardless of + // the identifier left a standard type such as com.adobe.pdf unnamed on those releases, so a + // drag of one was neither discovered while it hovered nor materialized when it dropped -- + // the outgoing direction had its legacy conversion and the incoming one did not. + NSString* legacy = cn1LegacyMimeForUti(uti); + if (legacy != nil) { + return legacy; + } // A dynamic or private identifier with no MIME equivalent. Naming it anyway would fill the // content with identifiers no drop target could match on. return nil; diff --git a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java index 36552425be0..f3595b74825 100644 --- a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java +++ b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java @@ -9296,9 +9296,13 @@ public static void nativeDropAddCallback(String mimeType, String text, byte[] bi pendingDrop.setFiles(split(text)); return; } - if (binary != null && binary.length > 0) { + // Length is not a test of presence. A representation the drag advertised and that is + // legitimately empty -- an empty string, a zero byte payload -- was discarded here, so + // the materialized content lacked a type the hover had accepted and the drop was then + // refused by the very target that agreed to take it. Null is absent; empty is present. + if (binary != null) { pendingDrop.setData(mimeType, binary); - } else if (text != null && text.length() > 0) { + } else if (text != null) { pendingDrop.setData(mimeType, text); } } @@ -9323,11 +9327,20 @@ public Object getClipboardData(String requested) { if (in == null) { return null; } + byte[] bytes; try { - return com.codename1.io.Util.readInputStream(in); + bytes = com.codename1.io.Util.readInputStream(in); } finally { in.close(); } + // A text type reads back as text. A document provider from Files commonly + // offers a plain text representation beside its file URL, and answering + // that with bytes made getText() and NativeDropEvent.getText() null for a + // type the drop had just accepted. + if (bytes != null && requested != null && requested.startsWith("text/")) { + return new String(bytes, "UTF-8"); + } + return bytes; } catch (Throwable err) { com.codename1.io.Log.e(err); return null; From 25f144c4eb5e29cabfc387b85d6673ba2fac97e5 Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 17:33:35 +0300 Subject: [PATCH 13/26] Review: a refusal undone twice over, a payload that outlives its gesture Four, and two of them were made by the fixes of the last two rounds. **A queued drag-over undid a refusal made in enter.** The callbacks are queued onto the event dispatch thread, so an over event can be queued before the enter event ahead of it has run -- and the starting action was captured when the event was queued, so the over event then restored the default over whatever the enter callback had since decided. A target that rejects only in nativeDragEnter had its rejection undone by the very next no-op nativeDragOver and was handed the drop. The starting action is read as the callback runs now. **Making iOS lazy left its payload unreachable.** An item provider's load handler is asynchronous by design and a receiving application may defer reading a representation until after the session has ended -- at which point dragCompleted has cleared the active drag and the lookup answered with nothing. The exported operation is now held independently of the gesture until the next drag replaces it. Deferring the work meant keeping it alive longer than the gesture, and the previous round did only the first half of that. **The desktop modifier fix left a stale cached action.** Narrowing the permitted set to the modifier's choice means an action agreed under the old set may no longer be on offer, and the same-target path returned it unchanged. It is revalidated against the current set now. **And PNG bytes were filed under image/jpeg.** A decoded java.awt.Image can only be produced as a PNG, so PNG is what it advertises; filing PNG bytes under whatever the flavor called itself handed a target bytes it could not decode by the type it asked for. Third port this has happened on -- iOS, then Android, now the desktop. The interesting part is the interaction. Revalidating a cached action recomputes anything not in the permitted set, and ACTION_NONE trivially is not -- so the second fix above resurrected the refusals the first one exists to protect, which is the same defect arriving from the other direction. Both would have shipped looking right. ACTION_NONE is excluded now, being a decision rather than a stale value, and the test covers both routes: it fails with the queue-time capture restored and it failed with the resurrection present, so it passes only while both hold. Co-Authored-By: Claude Opus 5 (1M context) --- .../com/codename1/ui/NativeDragAndDrop.java | 32 +++++++++++--- .../impl/javase/JavaSENativeDragAndDrop.java | 9 ++++ .../codename1/impl/ios/IOSImplementation.java | 14 +++++- .../codename1/ui/NativeDragAndDropTest.java | 44 +++++++++++++++++++ 4 files changed, 93 insertions(+), 6 deletions(-) diff --git a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java index e72e185c393..2a1bc477018 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java @@ -525,9 +525,23 @@ public static int dragOver(int windowId, int x, int y, ClipboardContent content, overDispatchPending = false; currentAction = target == null ? NativeDragOperation.ACTION_NONE : preferredAction(allowedActions & target.getAcceptedDropActions()); - } else if (target != null && !overDispatchPending) { - overDispatchPending = true; - dispatchOver = true; + } else if (target != null) { + if (currentAction != NativeDragOperation.ACTION_NONE + && (currentAction & allowedActions) == 0) { + // The permitted set changed while the pointer stayed put, which is what the + // desktop modifier does. An action agreed under the old set is no longer on + // offer, so keeping it told the platform something it had just withdrawn. + // + // ACTION_NONE is excluded deliberately: it is a decision, not a stale value. + // Recomputing it here turned a target's refusal back into the default and + // handed it the drop -- the same defect the enter callback's own rejection + // suffered, arriving by a different route. + currentAction = preferredAction(allowedActions & target.getAcceptedDropActions()); + } + if (!overDispatchPending) { + overDispatchPending = true; + dispatchOver = true; + } } answer = target == null ? NativeDragOperation.ACTION_NONE : currentAction; } @@ -754,11 +768,9 @@ private static void dispatch(final Component target, final ActionEvent.Type type return; } final boolean local; - final int startingAction; final int generation; synchronized (LOCK) { local = active != null; - startingAction = currentAction; generation = targetGeneration; } Display.getInstance().callSerially(new Runnable() { @@ -767,6 +779,16 @@ public void run() { try { NativeDropEvent ev = new NativeDropEvent(target, type, content, x, y, allowedActions, local); if (type == ActionEvent.Type.NativeDragOver || type == ActionEvent.Type.NativeDragEnter) { + // Read as this runs, not when it was queued. A drag event can arrive + // before the enter callback ahead of it in the queue has run, and + // capturing the answer at queueing time meant this event then restored + // the default over a decision that callback had since made -- so a + // target rejecting only in nativeDragEnter had its rejection undone by + // the very next no-op nativeDragOver, and was handed the drop. + int startingAction; + synchronized (LOCK) { + startingAction = currentAction; + } // The target starts from what the framework already agreed to, so a // target that does not care keeps the answer stable instead of // resetting it to the default on every event. diff --git a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java index c92be2370a7..65a029d9de1 100644 --- a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java +++ b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java @@ -270,6 +270,15 @@ private static String mimeFor(DataFlavor flavor) { if (DataFlavor.imageFlavor.equals(flavor)) { return ClipboardContent.MIME_PNG; } + if (flavor.getRepresentationClass() != null + && java.awt.Image.class.isAssignableFrom(flavor.getRepresentationClass())) { + // A decoded image, whatever the flavor calls itself. All this can produce from one + // is a PNG, so PNG is what it advertises -- filing PNG bytes under image/jpeg or + // image/webp because the flavor said so handed a target bytes it could not decode + // by the type it had asked for. A source offering the real encoded bytes offers + // them through a stream flavor as well, and that one keeps its own type. + return ClipboardContent.MIME_PNG; + } if (DataFlavor.stringFlavor.equals(flavor)) { return ClipboardContent.MIME_TEXT; } diff --git a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java index f3595b74825..b3f4e212d79 100644 --- a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java +++ b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java @@ -9271,6 +9271,15 @@ public static void nativeDragExitCallback() { NativeDragAndDrop.dragExit(0); } + /// The operation whose representations the item providers may still be asked for. + /// + /// Deliberately not the active drag: an item provider's load handler is asynchronous by + /// design, and a receiving application is free to defer reading a representation until + /// after the session has ended -- by which point the active drag has been cleared and the + /// lookup would answer with nothing at all. Held until the next drag replaces it, which is + /// one payload and the price of letting a receiver read late. + private static NativeDragOperation exportedDrag; + /// The drop being assembled by CN1DragAndDrop.m, one representation at a time. /// /// Only ever touched from the three callbacks below, which UIKit runs in order on the main @@ -9366,6 +9375,7 @@ public static int nativeDragSessionStartedCallback() { if (op == null) { return 0; } + exportedDrag = op; ClipboardContent content = op.getContent(); nativeInstance.beginNativeDragPayload(); // Names, not values. Reading a representation here would build every promised file and @@ -9400,7 +9410,9 @@ public static int nativeDragSessionStartedCallback() { /// /// its bytes, or null when the operation cannot supply it public static byte[] nativeDragResolveCallback(String mimeType) { - NativeDragOperation op = NativeDragAndDrop.getActiveDrag(); + // exportedDrag, not the active drag: see the field. A receiver that reads a + // representation after the session has ended still gets it. + NativeDragOperation op = exportedDrag; if (op == null || mimeType == null || mimeType.length() == 0) { return null; } diff --git a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java index 88c401c1c64..b6694782631 100644 --- a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java +++ b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java @@ -289,6 +289,50 @@ void dropDeliversTheContentAndNotifiesTheListener() { assertFalse(seen[0].isLocal(), "a drag this application did not start is not local"); } + @FormTest + void aRejectionInEnterSurvivesAnAlreadyQueuedOver() { + Form form = Display.getInstance().getCurrent(); + final List seen = new ArrayList(); + // Decides in nativeDragEnter and nowhere else, which is the case the queued over event + // used to undo. + Container target = new Container() { + @Override + protected void nativeDragEnter(NativeDropEvent ev) { + seen.add("enter"); + ev.reject(); + } + + @Override + protected void nativeDrop(NativeDropEvent ev) { + seen.add("drop"); + } + }; + target.setNativeDropTarget(true); + form.setLayout(new BorderLayout()); + form.add(BorderLayout.CENTER, target); + form.revalidate(); + + int x = target.getAbsoluteX() + 5; + int y = target.getAbsoluteY() + 5; + NativeDragAndDrop.dragEnter(0, x, y, textContent("hi"), NativeDragOperation.ACTION_COPY); + // A second motion event before the queue drains, so its callback is queued behind the + // enter that has not run yet. + NativeDragAndDrop.dragOver(0, x + 1, y + 1, textContent("hi"), NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + + assertEquals(NativeDragOperation.ACTION_NONE, + NativeDragAndDrop.dragOver(0, x + 2, y + 2, textContent("hi"), + NativeDragOperation.ACTION_COPY), + "the refusal made in nativeDragEnter must not be undone by an over event that " + + "was queued before it ran"); + assertEquals(NativeDragOperation.ACTION_NONE, + NativeDragAndDrop.drop(0, x, y, textContent("hi"), NativeDragOperation.ACTION_COPY)); + flushSerialCalls(); + assertFalse(seen.contains("drop")); + NativeDragAndDrop.dragExit(0); + flushSerialCalls(); + } + @FormTest void aRejectionMadeWhileHoveringSurvivesTheDrop() { Form form = Display.getInstance().getCurrent(); From e819b12260455b871abc98b6595bd8b072e67dd8 Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 18:23:10 +0300 Subject: [PATCH 14/26] Review: a target that could do nothing, a stale picture, and bytes from the wrong drag **A target that could perform none of the offered actions still swallowed the drag.** findTarget chose on the content alone, so a move-only target nested in a copy-capable one was picked for a copy-only drag, answered with nothing, and the ancestor that would have taken the drop was never reached. Refusing on the action is the same kind of refusal as refusing on the MIME type, and the walk treats it the same way now -- which is what the documented "a target that refuses a particular payload lets an ancestor have it" always meant. **A generated preview became the operation's permanent image.** The snapshot the framework renders from the source component was written in as though the application had supplied it, and the operation is reused for every drag of its component -- so every later drag showed the first one's picture, taken before whatever the component has done since, grabbed at a point the user did not press. Generated images are marked as such and re-rendered per gesture; an image the application supplied is still never touched. **An iOS provider could load bytes from the wrong drag.** This one is the third consequence in a chain: making the payload lazy meant it had to outlive the gesture, and outliving the gesture in a single global meant a load handler from an earlier drag resolved against whatever was being dragged by the time it ran. Each session now carries an id its handlers keep, and the two most recent are held. A read older than that answers null, because handing a receiver another drag's bytes is worse than handing it none. **And a file plus a text fallback was two dragged objects.** UIKit exposes each UIDragItem as its own thing, so a receiver could import the document and a stray piece of text rather than choosing the best form of one object. The alternatives attach to the file item's own provider now. Both core fixes have tests, probed by disabling both at once: each fails on its own account, and the target-walk change takes four more with it, which is what a change to target selection should do. Co-Authored-By: Claude Opus 5 (1M context) --- .../com/codename1/ui/NativeDragAndDrop.java | 44 +++++++---- .../com/codename1/ui/NativeDragOperation.java | 24 ++++++ Ports/iOSPort/nativeSources/CN1DragAndDrop.h | 16 ++-- Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 68 +++++++++++------ Ports/iOSPort/nativeSources/IOSNative.m | 10 +-- .../codename1/impl/ios/IOSImplementation.java | 37 ++++++--- .../src/com/codename1/impl/ios/IOSNative.java | 14 +++- .../codename1/ui/NativeDragAndDropTest.java | 76 +++++++++++++++++++ 8 files changed, 228 insertions(+), 61 deletions(-) diff --git a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java index 2a1bc477018..6a98cf0fee5 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java @@ -296,13 +296,13 @@ static void pressedOn(Component cmp, int x, int y) { if (op != null) { op.setSource(source); try { - if (op.getDragImage() == null && Display.impl.isNativeDragImageNeededOnPrepare()) { + if (needsGeneratedImage(op) && Display.impl.isNativeDragImageNeededOnPrepare()) { // The platform asks for the preview from inside its own gesture callback, // which is not a moment at which a component can be rendered. Rendering here // costs a snapshot per press on a drag source, which is what the lightweight // drag has always cost when one starts. - op.setDragImage(source.getDragImage()); - op.setDragImageOffset(x - source.getAbsoluteX(), y - source.getAbsoluteY()); + op.setGeneratedDragImage(source.getDragImage(), + x - source.getAbsoluteX(), y - source.getAbsoluteY()); } } catch (Throwable err) { Log.e(err); @@ -384,13 +384,14 @@ static boolean pointerDragged(int x, int y) { grabX = pressX; grabY = pressY; } - if (op.getDragImage() == null && source != null) { + if (needsGeneratedImage(op) && source != null) { try { - op.setDragImage(source.getDragImage()); - // Only when the image is the one we just rendered from the component. An - // application that supplied its own image may also have positioned it, and + // Rendered afresh for this gesture, and recorded as generated rather than + // written in as though the application had supplied it. An application's own + // image is never touched -- it may have been positioned deliberately, and // overwriting that offset would tear the image away from the pointer. - op.setDragImageOffset(grabX - source.getAbsoluteX(), grabY - source.getAbsoluteY()); + op.setGeneratedDragImage(source.getDragImage(), + grabX - source.getAbsoluteX(), grabY - source.getAbsoluteY()); } catch (Throwable err) { Log.e(err); } @@ -437,6 +438,13 @@ static void pointerReleased() { } } + /// True when this gesture should render its own preview: either the operation has no image + /// at all, or the one it has was rendered for an earlier drag of the same reusable + /// operation and is now out of date. + private static boolean needsGeneratedImage(NativeDragOperation op) { + return op.getDragImage() == null || op.isDragImageGenerated(); + } + private static int dragThreshold() { try { return Math.max(4, Display.getInstance().convertToPixels(DRAG_THRESHOLD_MM)); @@ -508,7 +516,7 @@ public static int dragEnter(int windowId, int x, int y, ClipboardContent content /// /// the action a drop would perform right now, or `NativeDragOperation#ACTION_NONE` public static int dragOver(int windowId, int x, int y, ClipboardContent content, int allowedActions) { - Component target = findTarget(windowId, x, y, content); + Component target = findTarget(windowId, x, y, content, allowedActions); Component previous; boolean changed; boolean dispatchOver = false; @@ -594,7 +602,7 @@ public static void dragExit(int windowId) { /// the action actually accepted, or `NativeDragOperation#ACTION_NONE` when nothing under /// the pointer took the drop and the port should report the transfer as failed public static int drop(int windowId, int x, int y, ClipboardContent content, int action) { - Component target = findTarget(windowId, x, y, content); + Component target = findTarget(windowId, x, y, content, action); int accepted; synchronized (LOCK) { if (target != null && target == currentTarget) { // NOPMD CompareObjectsWithEquals @@ -652,7 +660,7 @@ public static int drop(int windowId, int x, int y, ClipboardContent content, int /// the action the drop would perform, or `NativeDragOperation#ACTION_NONE` public static int plannedDropAction(int windowId, int x, int y, ClipboardContent content, int action) { - Component target = findTarget(windowId, x, y, content); + Component target = findTarget(windowId, x, y, content, action); synchronized (LOCK) { if (target != null && target == currentTarget) { // NOPMD CompareObjectsWithEquals return currentAction; @@ -697,7 +705,16 @@ public void run() { /// /// Runs on the native drag thread and reads the component tree without mutating it, which /// is the same thing the ports already do to route a native pointer press. - private static Component findTarget(int windowId, int x, int y, ClipboardContent content) { + /// #### Parameters + /// + /// - `actions`: the actions in play, so a target that can perform none of them is passed + /// over rather than selected and then found to have nothing to offer. A move-only target + /// nested in a copy-capable one used to swallow a copy-only drag: it was chosen on the + /// content alone, answered with nothing, and the ancestor that would have taken the drop + /// was never reached. Refusing on the action is the same kind of refusal as refusing on + /// the MIME type, and the walk treats it the same way. + private static Component findTarget(int windowId, int x, int y, ClipboardContent content, + int actions) { Container root = surfaceFor(windowId); if (root == null) { return null; @@ -711,7 +728,8 @@ private static Component findTarget(int windowId, int x, int y, ClipboardContent return null; } while (cmp != null) { - if (cmp.isNativeDropTarget() && !cmp.isIgnorePointerEvents() && cmp.isEnabled()) { + if (cmp.isNativeDropTarget() && !cmp.isIgnorePointerEvents() && cmp.isEnabled() + && (actions & cmp.getAcceptedDropActions()) != 0) { try { if (cmp.canAcceptNativeDrop(content)) { return cmp; diff --git a/CodenameOne/src/com/codename1/ui/NativeDragOperation.java b/CodenameOne/src/com/codename1/ui/NativeDragOperation.java index d4734443461..532b4d6461e 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragOperation.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragOperation.java @@ -72,6 +72,9 @@ public class NativeDragOperation { private int dragImageOffsetY; private String label; private int performedAction = ACTION_NONE; + /// True when the image below is the one the framework rendered from the source component, + /// rather than one the application supplied. + private boolean dragImageGenerated; private Component source; private EventDispatcher completionListeners; @@ -154,9 +157,30 @@ public Image getDragImage() { /// this instance, for chaining public NativeDragOperation setDragImage(Image dragImage) { this.dragImage = dragImage; + this.dragImageGenerated = false; return this; } + /// Installs the image the framework rendered from the source component, and the point of it + /// the press landed on. + /// + /// Kept apart from `#setDragImage(com.codename1.ui.Image)` because the operation is reused + /// for every drag of its component: writing a generated snapshot in as though the + /// application had supplied it meant every later drag showed the first one's picture, taken + /// before whatever the component has done since, grabbed at a point the user did not press. + void setGeneratedDragImage(Image image, int offsetX, int offsetY) { + this.dragImage = image; + this.dragImageOffsetX = offsetX; + this.dragImageOffsetY = offsetY; + this.dragImageGenerated = true; + } + + /// True when the current image was rendered by the framework, so a later gesture should + /// render a fresh one rather than reuse it. + boolean isDragImageGenerated() { + return dragImageGenerated; + } + /// Returns the x offset of the cursor within the drag image. public int getDragImageOffsetX() { return dragImageOffsetX; diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.h b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h index 9120e37268c..e07c4466f3e 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.h +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h @@ -89,9 +89,13 @@ void CN1EnableNativeDropTarget(void); void CN1PrepareNativeDrag(NSString* mimeTypes, int allowedActions, NSData* dragImagePng, int touchX, int touchY); -/// Clears the payload, ready for the representations of the session UIKit has just started. -/// Called from Java, from inside the session-started callback below. -void CN1BeginNativeDragPayload(void); +/// Clears the payload, ready for the representations of the session UIKit has just started, +/// and records the id the framework has given that session. +/// +/// The id travels with every load handler this session registers, so a representation read +/// late -- after the user has begun another drag -- is resolved against the operation it +/// belongs to rather than whatever is being dragged by then. +void CN1BeginNativeDragPayload(int sessionId); /// Names a representation the drag can offer without producing it. /// @@ -146,9 +150,9 @@ void CN1NativeDragDeliverDropAddFile(NSString* mimeType, NSString* path); /// it. int CN1NativeDragDeliverDropCommit(int x, int y, int action); -/// Produces one representation of the drag in progress, on demand. Returns nil when the -/// operation cannot supply it. -NSData* CN1NativeDragDeliverResolve(NSString* mimeType); +/// Produces one representation of a drag on demand. Returns nil when the operation that +/// session belongs to can no longer supply it. +NSData* CN1NativeDragDeliverResolve(NSString* mimeType, int sessionId); /// Announces that UIKit has started a drag session. Returns the actions the framework's staged /// operation allows, or 0 when it has none -- in which case no drag begins. The Java side calls diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index 9eaa76cdd06..df672386cd9 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -66,7 +66,7 @@ void CN1PrepareNativeDrag(NSString* mimeTypes, int allowedActions, NSData* dragI int touchX, int touchY) { } -void CN1BeginNativeDragPayload(void) { +void CN1BeginNativeDragPayload(int sessionId) { } void CN1DeclareNativeDragPayload(NSString* mimeType) { @@ -102,6 +102,10 @@ void CN1CancelNativeDrag(void) { static NSMutableArray* cn1DragMimes = nil; static NSMutableArray* cn1DragFileUrls = nil; +/// The framework's id for the session being built, captured by every load handler it registers +/// so a late read resolves against its own operation and not the next drag's. +static int cn1DragSessionId = 0; + /// The last action the framework agreed to, reused when a drop arrives without one. static int cn1LastDropAction = CN1_DND_ACTION_NONE; @@ -387,13 +391,14 @@ void CN1PrepareNativeDrag(NSString* mimeTypes, int allowedActions, NSData* dragI #endif } -void CN1BeginNativeDragPayload(void) { +void CN1BeginNativeDragPayload(int sessionId) { #ifndef CN1_USE_ARC [cn1DragMimes release]; [cn1DragFileUrls release]; #endif cn1DragMimes = [[NSMutableArray alloc] init]; cn1DragFileUrls = [[NSMutableArray alloc] init]; + cn1DragSessionId = sessionId; } void CN1DeclareNativeDragPayload(NSString* mimeType) { @@ -463,13 +468,48 @@ @implementation CN1DragAndDropDelegate cn1LocalDropResult = -1; NSMutableArray* items = [NSMutableArray array]; + const int sessionId = cn1DragSessionId; + + // Every representation the operation declared, registered lazily. The value is fetched when + // a receiver reads that type, not now: a drag that is begun and abandoned must not have + // written the file or encoded the image it was merely offering. NSItemProvider allows a + // load handler to answer asynchronously, which is what lets the fetch happen on the main + // thread where every other call into the framework from this file happens. Nothing is + // retained by hand either -- copying a block retains what it captures, and an explicit + // retain here leaked the whole payload of every drag. + void (^registerDeclared)(NSItemProvider*) = ^(NSItemProvider* provider) { + for (NSString* mime in cn1DragMimes) { + NSString* uti = cn1UtiForMime(mime); + if (uti == nil) { + continue; + } + [provider registerDataRepresentationForTypeIdentifier:uti + visibility:NSItemProviderRepresentationVisibilityAll + loadHandler:^NSProgress *(void (^completion)(NSData *, NSError *)) { + dispatch_async(dispatch_get_main_queue(), ^{ + completion(CN1NativeDragDeliverResolve(mime, sessionId), nil); + }); + return nil; + }]; + } + }; + // Files first, one item each: a receiver that copies documents expects one item per // document, and collapsing several into one loses all but the first. + BOOL declaredAttached = NO; for (NSURL* url in cn1DragFileUrls) { NSItemProvider* provider = [[NSItemProvider alloc] initWithContentsOfURL:url]; if (provider == nil) { continue; } + if (!declaredAttached) { + // The other representations belong to this same object, not to one of their own. + // Given a file and a text fallback, adding a second item made UIKit expose them as + // two dragged things, so a receiver could import the document *and* a stray piece + // of text instead of choosing the best form of one. + registerDeclared(provider); + declaredAttached = YES; + } UIDragItem* item = [[UIDragItem alloc] initWithItemProvider:provider]; [items addObject:item]; #ifndef CN1_USE_ARC @@ -477,29 +517,9 @@ @implementation CN1DragAndDropDelegate [item release]; #endif } - if (cn1DragMimes.count > 0) { + if (!declaredAttached && cn1DragMimes.count > 0) { NSItemProvider* provider = [[NSItemProvider alloc] init]; - for (NSString* mime in cn1DragMimes) { - NSString* uti = cn1UtiForMime(mime); - if (uti == nil) { - continue; - } - // The value is fetched when a receiver reads this type, not now: a drag that is - // begun and abandoned must not have written the file or encoded the image it was - // merely offering. NSItemProvider allows a load handler to answer asynchronously, - // which is what lets the fetch happen on the main thread where every other call - // into the framework from this file happens. No retain of the captured strings - // either -- copying a block retains what it captures, and an explicit retain here - // leaked the whole payload of every drag. - [provider registerDataRepresentationForTypeIdentifier:uti - visibility:NSItemProviderRepresentationVisibilityAll - loadHandler:^NSProgress *(void (^completion)(NSData *, NSError *)) { - dispatch_async(dispatch_get_main_queue(), ^{ - completion(CN1NativeDragDeliverResolve(mime), nil); - }); - return nil; - }]; - } + registerDeclared(provider); UIDragItem* item = [[UIDragItem alloc] initWithItemProvider:provider]; [items addObject:item]; #ifndef CN1_USE_ARC diff --git a/Ports/iOSPort/nativeSources/IOSNative.m b/Ports/iOSPort/nativeSources/IOSNative.m index b0236fb70c2..1ef1f637e54 100644 --- a/Ports/iOSPort/nativeSources/IOSNative.m +++ b/Ports/iOSPort/nativeSources/IOSNative.m @@ -1081,8 +1081,8 @@ void com_codename1_impl_ios_IOSNative_prepareNativeDrag___java_lang_String_int_b POOL_END(); } -void com_codename1_impl_ios_IOSNative_beginNativeDragPayload__(CN1_THREAD_STATE_MULTI_ARG JAVA_OBJECT instanceObject) { - CN1BeginNativeDragPayload(); +void com_codename1_impl_ios_IOSNative_beginNativeDragPayload___int(CN1_THREAD_STATE_MULTI_ARG JAVA_OBJECT instanceObject, JAVA_INT sessionId) { + CN1BeginNativeDragPayload((int)sessionId); } void com_codename1_impl_ios_IOSNative_declareNativeDragPayload___java_lang_String(CN1_THREAD_STATE_MULTI_ARG JAVA_OBJECT instanceObject, JAVA_OBJECT mimeType) { @@ -1143,9 +1143,9 @@ void CN1NativeDragDeliverDropAddFile(NSString* mimeType, NSString* path) { fromNSString(CN1_THREAD_GET_STATE_PASS_ARG path)); } -NSData* CN1NativeDragDeliverResolve(NSString* mimeType) { - JAVA_OBJECT bytes = com_codename1_impl_ios_IOSImplementation_nativeDragResolveCallback___java_lang_String_R_byte_1ARRAY( - CN1_THREAD_GET_STATE_PASS_ARG fromNSString(CN1_THREAD_GET_STATE_PASS_ARG mimeType)); +NSData* CN1NativeDragDeliverResolve(NSString* mimeType, int sessionId) { + JAVA_OBJECT bytes = com_codename1_impl_ios_IOSImplementation_nativeDragResolveCallback___java_lang_String_int_R_byte_1ARRAY( + CN1_THREAD_GET_STATE_PASS_ARG fromNSString(CN1_THREAD_GET_STATE_PASS_ARG mimeType), sessionId); return bytes == JAVA_NULL ? nil : arrayToData(bytes); } diff --git a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java index b3f4e212d79..0c8d8033e74 100644 --- a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java +++ b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java @@ -9271,14 +9271,23 @@ public static void nativeDragExitCallback() { NativeDragAndDrop.dragExit(0); } - /// The operation whose representations the item providers may still be asked for. + /// The two most recent exported drags, each under the session id its load handlers carry. /// /// Deliberately not the active drag: an item provider's load handler is asynchronous by /// design, and a receiving application is free to defer reading a representation until /// after the session has ended -- by which point the active drag has been cleared and the - /// lookup would answer with nothing at all. Held until the next drag replaces it, which is - /// one payload and the price of letting a receiver read late. + /// lookup would answer with nothing at all. + /// + /// Two of them, and matched by id, because a single slot answered a late read with + /// *whatever was being dragged by then*: a handler from the previous drag resolving after + /// the next one began produced that one's bytes. Wrong data is worse than none, so a read + /// older than these answers null instead. Two payloads is the bound: a receiver would have + /// to sit on an unread representation across two further complete drags to fall off it. private static NativeDragOperation exportedDrag; + private static int exportedDragId; + private static NativeDragOperation previousExportedDrag; + private static int previousExportedDragId; + private static int nextDragSessionId; /// The drop being assembled by CN1DragAndDrop.m, one representation at a time. /// @@ -9375,9 +9384,12 @@ public static int nativeDragSessionStartedCallback() { if (op == null) { return 0; } + previousExportedDrag = exportedDrag; + previousExportedDragId = exportedDragId; exportedDrag = op; + exportedDragId = ++nextDragSessionId; ClipboardContent content = op.getContent(); - nativeInstance.beginNativeDragPayload(); + nativeInstance.beginNativeDragPayload(exportedDragId); // Names, not values. Reading a representation here would build every promised file and // encode every promised image at the moment the drag begins -- including for a drag the // user then abandons -- which is the opposite of what setDataProvider promises. The @@ -9406,13 +9418,20 @@ public static int nativeDragSessionStartedCallback() { /// /// - `mimeType`: the representation being read /// + /// - `sessionId`: the drag the reading item provider belongs to + /// /// #### Returns /// - /// its bytes, or null when the operation cannot supply it - public static byte[] nativeDragResolveCallback(String mimeType) { - // exportedDrag, not the active drag: see the field. A receiver that reads a - // representation after the session has ended still gets it. - NativeDragOperation op = exportedDrag; + /// its bytes, or null when that drag can no longer supply it + public static byte[] nativeDragResolveCallback(String mimeType, int sessionId) { + // Matched by session id, not simply the latest: a handler from an earlier drag must + // answer for that drag or not at all. See the fields. + NativeDragOperation op = null; + if (exportedDrag != null && sessionId == exportedDragId) { + op = exportedDrag; + } else if (previousExportedDrag != null && sessionId == previousExportedDragId) { + op = previousExportedDrag; + } if (op == null || mimeType == null || mimeType.length() == 0) { return null; } diff --git a/Ports/iOSPort/src/com/codename1/impl/ios/IOSNative.java b/Ports/iOSPort/src/com/codename1/impl/ios/IOSNative.java index 090025daff4..399e9d5a12e 100644 --- a/Ports/iOSPort/src/com/codename1/impl/ios/IOSNative.java +++ b/Ports/iOSPort/src/com/codename1/impl/ios/IOSNative.java @@ -375,10 +375,16 @@ native void fillGradient(int kind, int stopCount, float[] positions, float[] pre native void prepareNativeDrag(String mimeTypes, int allowedActions, byte[] dragImagePng, int touchX, int touchY); - /// Clears the payload, ready for the representations of the session UIKit has just started. - /// Called from inside the session-started callback, so that a promised representation is - /// built only once a drag really happens. - native void beginNativeDragPayload(); + /// Clears the payload, ready for the representations of the session UIKit has just started, + /// and records the id the framework gave that session. Called from inside the + /// session-started callback, so that a promised representation is built only once a drag + /// really happens. + /// + /// #### Parameters + /// + /// - `sessionId`: travels with every load handler this session registers, so a + /// representation read after another drag has begun resolves against its own operation + native void beginNativeDragPayload(int sessionId); /// Names a representation the drag can offer, without producing it. /// diff --git a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java index b6694782631..8146493a023 100644 --- a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java +++ b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java @@ -357,6 +357,82 @@ void aRejectionMadeWhileHoveringSurvivesTheDrop() { "and no drop event is delivered, which is what reject() promises"); } + @FormTest + void aTargetThatCannotPerformTheActionLetsAnAncestorHaveIt() { + Form form = Display.getInstance().getCurrent(); + DropRecorder outer = addTarget(form); + outer.setAcceptedDropActions(NativeDragOperation.ACTION_COPY); + DropRecorder inner = new DropRecorder(); + inner.setNativeDropTarget(true); + inner.setAcceptedDropActions(NativeDragOperation.ACTION_MOVE); + outer.setLayout(new BorderLayout()); + outer.add(BorderLayout.CENTER, inner); + form.revalidate(); + + // A copy-only drag over a move-only target nested in a copy-capable one. Choosing the + // inner target on the content alone and only then finding it can do nothing swallowed + // the drag: refusing on the action is the same kind of refusal as refusing on the MIME + // type, and the walk has to treat it the same way. + int x = inner.getAbsoluteX() + 2; + int y = inner.getAbsoluteY() + 2; + assertEquals(NativeDragOperation.ACTION_COPY, + NativeDragAndDrop.dragEnter(0, x, y, textContent("hi"), + NativeDragOperation.ACTION_COPY), + "the copy-capable ancestor takes it"); + NativeDragAndDrop.drop(0, x, y, textContent("hi"), NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + assertTrue(outer.events.contains("drop")); + assertFalse(inner.events.contains("drop")); + } + + @FormTest + void eachGestureRendersItsOwnDragImage() { + implementation.resetNativeDragState(); + implementation.setNativeDragAndDropSupported(true); + try { + Form form = Display.getInstance().getCurrent(); + Container source = new Container(); + NativeDragOperation op = new NativeDragOperation("reused"); + source.setNativeDragOperation(op); + form.setLayout(new BorderLayout()); + form.add(BorderLayout.CENTER, source); + form.revalidate(); + + int x = source.getAbsoluteX() + 10; + int y = source.getAbsoluteY() + 10; + form.pointerPressed(x, y); + form.pointerDragged(x + 200, y + 200); + Image first = op.getDragImage(); + assertNotNull(first); + NativeDragAndDrop.dragCompleted(NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + + // The same reusable operation, dragged again. The generated snapshot must not have + // become the operation's permanent image: the component may look different now and + // the press landed somewhere else. + form.pointerPressed(x + 4, y + 4); + form.pointerDragged(x + 200, y + 200); + Image second = op.getDragImage(); + assertNotNull(second); + assertNotSame(first, second, + "a framework-rendered preview belongs to its gesture, not to the operation"); + NativeDragAndDrop.dragCompleted(NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + + // An image the application supplied is never replaced. + Image supplied = Image.createImage(4, 4); + op.setDragImage(supplied); + form.pointerPressed(x, y); + form.pointerDragged(x + 200, y + 200); + assertSame(supplied, op.getDragImage(), "an application's own image is left alone"); + NativeDragAndDrop.dragCompleted(NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + } finally { + implementation.setNativeDragAndDropSupported(false); + implementation.resetNativeDragState(); + } + } + @FormTest void dropOnNothingReportsFailureSoThePortCanTellTheSource() { Form form = Display.getInstance().getCurrent(); From 4353c418cfbec3043479dc112e347eb6e8bb42f5 Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 19:25:17 +0300 Subject: [PATCH 15/26] Review: an invisible source, a flavor nothing could serve, a value replaced, a payload evicted **A source the lightweight drag hid stayed hidden.** Where the operating system owns the drag gesture, startNativeDrag refuses and the gesture carries on as a lightweight drag -- which hides the component and draws its image -- until the platform's own recognizer fires. cancelLightweightDrag then cleared the drag state and left the component invisible, because dragFinishedImpl, the only other thing that makes it visible again, never runs on that path. Removing the fix fails the new test on its own. **The standard image flavor was advertised for encodings nothing could read.** Any image/* type added DataFlavor.imageFlavor, but serving it read only PNG, JPEG and GIF, so a WebP payload threw UnsupportedFlavorException -- and desktop receivers commonly pick that flavor ahead of the MIME specific stream, losing a drop whose bytes were perfectly readable. The flavor is now advertised only when ImageIO has a reader for the type, which is a question about the MIME type and so still builds no value, and it serves whatever it was advertised for rather than three named encodings. **Android replaced an application's own text with the plain fallback.** The exporter puts a text representation whose value differs from the carried text behind a content URI, exactly as it does binary; the synthesized extension has no MimeTypeMap entry, so the resolver reports octet-stream and the type came back reconstructed from the fallback instead of from its URI. Rather than widen the one-to-one guess, the pairing is now exact: the extension is what extensionForMime produced for the type that was written, so inverting the file name recovers the association the resolver lost -- and more than one unnameable representation survives the round trip. The count based pairing remains for clips this application did not write, and text from the carried text is applied last, only to what no URI accounted for. **An iOS payload was dropped while a provider could still read it.** Holding the two most recent drags was still a bound, and an NSItemProvider a receiver keeps has no bound: read it after two newer drags and it answered nothing. The providers now say when they are done. Every load handler captures a token for its session -- copying a block retains what it captures -- and the token's dealloc reports the release, so a payload lives exactly as long as something can ask for it. In the ordinary case that frees sooner than two slots did. One thing found while wiring that up: an operation permitting no actions was refused when a press staged it but not by startDrag or dragSessionStarted, so a late recognizer could make active a session that nothing would ever complete -- and a running drag is what stops the next one from starting. Guarded in the router rather than in the port, since every port has the same hole. Removing that guard alone fails eighteen of the twenty-nine core tests, all of them wedged. Co-Authored-By: Claude Opus 5 (1M context) --- .../src/com/codename1/ui/Component.java | 6 + .../com/codename1/ui/NativeDragAndDrop.java | 13 +- .../com/codename1/ui/NativeDragOperation.java | 3 + .../impl/android/AndroidImplementation.java | 120 ++++++++++++++++-- .../com/codename1/impl/javase/JavaSEPort.java | 77 +++++++++-- Ports/iOSPort/nativeSources/CN1DragAndDrop.h | 10 ++ Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 47 ++++++- Ports/iOSPort/nativeSources/IOSNative.m | 5 + .../codename1/impl/ios/IOSImplementation.java | 55 +++++--- .../codename1/ui/NativeDragAndDropTest.java | 74 +++++++++++ .../javase/JavaSENativeDragAndDropTest.java | 48 +++++++ 11 files changed, 407 insertions(+), 51 deletions(-) diff --git a/CodenameOne/src/com/codename1/ui/Component.java b/CodenameOne/src/com/codename1/ui/Component.java index 195d22f12a0..ac375a300dc 100644 --- a/CodenameOne/src/com/codename1/ui/Component.java +++ b/CodenameOne/src/com/codename1/ui/Component.java @@ -5988,6 +5988,12 @@ void initDragAndDrop(int x, int y) { void cancelLightweightDrag() { Component leadParent = LeadUtil.leadParentImpl(this); if (leadParent.dragActivated) { + if (leadParent.dragAndDropInitialized) { + // pointerDragged hides the source while the framework carries its image; the + // native session draws its own preview and never runs dragFinishedImpl, so + // without this the component the user dragged stays invisible for good. + leadParent.setVisible(true); + } Form p = getComponentForm(); if (p != null) { p.setDraggedComponent(null); diff --git a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java index 6a98cf0fee5..d21b0aaa43e 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java @@ -161,7 +161,11 @@ public static boolean isDragOutsideApplicationSupported() { /// true when the operating system took the drag; false when the platform has no native drag /// and drop, refused to start a session, or is already running one public static boolean startDrag(Component source, NativeDragOperation op) { - if (op == null || !isSupported()) { + if (op == null || !isSupported() + || op.getAllowedActions() == NativeDragOperation.ACTION_NONE) { + // Allowing nothing to be done with a drag is having no drag, and a press stages one + // on the same terms. Running it anyway would put a session in flight that no target + // could ever accept. return false; } synchronized (LOCK) { @@ -218,7 +222,12 @@ public static NativeDragOperation dragSessionStarted() { return null; } op = pending; - if (op == null) { + if (op == null || op.getAllowedActions() == NativeDragOperation.ACTION_NONE) { + // As in startDrag. Refusing *before* the operation is made active also keeps a + // session that can never complete from wedging every drag after it: a running + // drag is what stops the next one from starting, and nothing would report this + // one finished. A press does not stage such an operation in the first place, so + // this is only reachable if the source changed its mind mid-gesture. return null; } source = pendingSource; diff --git a/CodenameOne/src/com/codename1/ui/NativeDragOperation.java b/CodenameOne/src/com/codename1/ui/NativeDragOperation.java index 532b4d6461e..8682061a8c1 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragOperation.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragOperation.java @@ -126,6 +126,9 @@ public int getAllowedActions() { /// Sets the bit set of actions the source is willing to allow. The receiver chooses one of /// them, usually influenced by the modifier keys the user is holding. /// + /// Allowing none of them is allowing nothing to be done with the drag, so no drag begins at + /// all: there is nothing a receiver could accept. + /// /// #### Parameters /// /// - `allowedActions`: any combination of `#ACTION_COPY`, `#ACTION_MOVE` and `#ACTION_LINK` diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java index d501f8721df..905cd37faac 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java @@ -10668,6 +10668,7 @@ public Object getClipboardData(String mimeType) { private void fillAdvertisedTypes(ClipboardContent content, ClipDescription description, String plain, List fileUris, List unnamedUris) { List unsatisfiedBinary = new ArrayList(); + List unsatisfiedText = new ArrayList(); for (int iter = 0; iter < description.getMimeTypeCount(); iter++) { String mime = description.getMimeType(iter); if (mime == null) { @@ -10677,9 +10678,6 @@ private void fillAdvertisedTypes(ClipboardContent content, ClipDescription descr if (content.hasMimeType(mime)) { continue; } - if (!mime.startsWith("text/") && !"text/uri-list".equals(mime)) { - unsatisfiedBinary.add(mime); - } if ("text/uri-list".equals(mime)) { if (!fileUris.isEmpty()) { StringBuilder uris = new StringBuilder(); @@ -10693,18 +10691,116 @@ private void fillAdvertisedTypes(ClipboardContent content, ClipDescription descr } continue; } - if (mime.startsWith("text/") && plain != null && plain.length() > 0) { - content.setData(mime, plain); + // A text type is *not* assumed to be the carried text here. The exporter writes a + // text representation whose value differs from that text into a content URI exactly + // as it writes binary, so assuming made a target asking for an application's own + // text format receive the plain fallback instead of the value it published. + if (mime.startsWith("text/")) { + unsatisfiedText.add(mime); + } else { + unsatisfiedBinary.add(mime); + } + } + List unclaimed = new ArrayList(unnamedUris); + for (int iter = unclaimed.size() - 1; iter >= 0; iter--) { + Uri uri = unclaimed.get(iter); + String named = mimeForUnnamedUri(uri, unsatisfiedBinary, unsatisfiedText); + if (named != null) { + content.setDataProvider(named, uriBytesProvider(uri)); + unsatisfiedBinary.remove(named); + unsatisfiedText.remove(named); + unclaimed.remove(iter); + } + } + if (unclaimed.size() == 1) { + // One representation the clip promised and could not produce, and one URI whose + // type Android could not name: the pairing cannot be anything else. A byte backed + // type is taken first because bytes can only have come from a URI, where a text one + // may also be another reading of the text the clip carries. With more of either it + // could be, and inventing an association would tell a target it has something it + // may not -- which is the failure this whole path exists to avoid -- so those are + // left absent and the target correctly refuses. + String only = null; + if (unsatisfiedBinary.size() == 1) { + only = unsatisfiedBinary.remove(0); + } else if (unsatisfiedBinary.isEmpty() && unsatisfiedText.size() == 1) { + only = unsatisfiedText.remove(0); + } + if (only != null) { + content.setDataProvider(only, uriBytesProvider(unclaimed.get(0))); + } + } + if (plain != null && plain.length() > 0) { + for (int iter = 0; iter < unsatisfiedText.size(); iter++) { + // What is left: an Android clip carries a single text payload, and a text type + // no URI accounted for is another name for that payload -- which is exactly how + // the exporter advertises a reading whose value *is* the carried text. + content.setData(unsatisfiedText.get(iter), plain); + } + } + } + + /// The type an untyped content URI was published as, recovered from the name of the file it + /// serves. + /// + /// ContentResolver could not name it -- MimeTypeMap has no entry for an application defined + /// type, so the FileProvider serving it reports octet-stream -- but the extension is still + /// exactly what `#extensionForMime(java.lang.String)` produced for the type that was + /// written, so the association the resolver lost is recoverable rather than guessed. That is + /// what lets more than one unnameable representation survive the round trip. An extension + /// two advertised types share answers nothing, as does a clip this application did not + /// write. + private String mimeForUnnamedUri(Uri uri, List binary, List text) { + String name = displayNameFor(uri); + if (name == null) { + return null; + } + int dot = name.lastIndexOf('.'); + if (dot < 0 || dot == name.length() - 1) { + return null; + } + String extension = name.substring(dot + 1).toLowerCase(); + String match = null; + for (int pass = 0; pass < 2; pass++) { + List candidates = pass == 0 ? binary : text; + for (int iter = 0; iter < candidates.size(); iter++) { + String candidate = candidates.get(iter); + if (extension.equals(extensionForMime(candidate))) { + if (match != null) { + return null; + } + match = candidate; + } } } - if (unsatisfiedBinary.size() == 1 && unnamedUris.size() == 1) { - // One type the clip promised and could not produce, and one URI whose type Android - // could not name: the pairing cannot be anything else. With more of either it could - // be, and inventing an association would tell a target it has something it may not - // -- which is the failure this whole path exists to avoid -- so those are left - // absent and the target correctly refuses. - content.setDataProvider(unsatisfiedBinary.get(0), uriBytesProvider(unnamedUris.get(0))); + return match; + } + + /// The file name behind a content URI, which is where the extension an exporter chose + /// survives. A provider that will not answer OpenableColumns still has the name in its path. + private String displayNameFor(Uri uri) { + Cursor cursor = null; + try { + cursor = getContext().getContentResolver().query(uri, + new String[]{android.provider.OpenableColumns.DISPLAY_NAME}, + null, null, null); + if (cursor != null && cursor.moveToFirst()) { + int column = cursor.getColumnIndex(android.provider.OpenableColumns.DISPLAY_NAME); + if (column >= 0) { + String name = cursor.getString(column); + if (name != null && name.length() > 0) { + return name; + } + } + } + } catch (Throwable t) { + // Fall through to the path below. + } finally { + if (cursor != null) { + cursor.close(); + } } + return uri.getLastPathSegment(); } public static MediaException createMediaException(int extra) { diff --git a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSEPort.java b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSEPort.java index 62f42fc8753..e246430160b 100644 --- a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSEPort.java +++ b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSEPort.java @@ -1976,7 +1976,12 @@ static final class RichTransferable implements Transferable { continue; } if (mime.startsWith("image/")) { - if (!available.contains(DataFlavor.imageFlavor)) { + // Only when something can actually decode this encoding. Desktop receivers + // commonly pick the standard image flavor ahead of the MIME specific stream, + // so advertising it for a WebP -- which ImageIO has no reader for -- lost the + // whole drop to an UnsupportedFlavorException for a payload that was + // perfectly readable as bytes. + if (hasImageReader(mime) && !available.contains(DataFlavor.imageFlavor)) { available.add(DataFlavor.imageFlavor); } addBinaryFlavor(available, mime); @@ -2015,15 +2020,62 @@ private static void addBinaryFlavor(ArrayList available, String mime } } - private static byte[] imageBytes(ClipboardContent data) { - byte[] b = data.getBytes(ClipboardContent.MIME_PNG); - if (b == null) { - b = data.getBytes(ClipboardContent.MIME_JPEG); + /// True when this JVM has an `ImageIO` reader for the encoding, which is the question + /// `DataFlavor#imageFlavor` really asks. Asked by MIME type, so it reads no value and a + /// lazily provided representation is not built in order to answer it. + private static boolean hasImageReader(String mime) { + try { + return ImageIO.getImageReadersByMIMEType(mime).hasNext(); + } catch (Throwable err) { + return false; } - if (b == null) { - b = data.getBytes(ClipboardContent.MIME_GIF); + } + + /// Decodes whichever image representation this content offers that `ImageIO` can read. + /// + /// The three encodings the framework names come first because they are what a Codename + /// One source publishes; anything else -- a BMP, a TIFF, whatever a reader plugin adds -- + /// is tried afterwards, so the standard image flavor serves the same set of types + /// `#hasImageReader(java.lang.String)` advertised it for. + private static java.awt.Image decodeImage(ClipboardContent data) { + java.awt.Image img = decodeImage(data, ClipboardContent.MIME_PNG); + if (img == null) { + img = decodeImage(data, ClipboardContent.MIME_JPEG); + } + if (img == null) { + img = decodeImage(data, ClipboardContent.MIME_GIF); + } + if (img != null) { + return img; + } + String[] mimeTypes = data.getMimeTypes(); + for (int i = 0; i < mimeTypes.length; i++) { + String mime = mimeTypes[i]; + if (!mime.startsWith("image/") || ClipboardContent.MIME_PNG.equals(mime) + || ClipboardContent.MIME_JPEG.equals(mime) + || ClipboardContent.MIME_GIF.equals(mime) || !hasImageReader(mime)) { + continue; + } + img = decodeImage(data, mime); + if (img != null) { + return img; + } + } + return null; + } + + private static java.awt.Image decodeImage(ClipboardContent data, String mime) { + byte[] bytes = data.getBytes(mime); + if (bytes == null) { + return null; + } + try { + return ImageIO.read(new ByteArrayInputStream(bytes)); + } catch (Throwable err) { + // A representation that does not decode is not the image flavor's answer; the + // next one, or the byte stream flavor, still is. + return null; } - return b; } /// Resolves the `application/x-file-list` representation (a single path/URI `String` or a @@ -2080,12 +2132,9 @@ public Object getTransferData(DataFlavor flavor) throws UnsupportedFlavorExcepti throw new UnsupportedFlavorException(flavor); } if (DataFlavor.imageFlavor.equals(flavor)) { - byte[] bytes = imageBytes(data); - if (bytes != null) { - java.awt.Image img = ImageIO.read(new ByteArrayInputStream(bytes)); - if (img != null) { - return img; - } + java.awt.Image img = decodeImage(data); + if (img != null) { + return img; } throw new UnsupportedFlavorException(flavor); } diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.h b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h index e07c4466f3e..f0306390c35 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.h +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h @@ -154,6 +154,16 @@ int CN1NativeDragDeliverDropCommit(int x, int y, int action); /// session belongs to can no longer supply it. NSData* CN1NativeDragDeliverResolve(NSString* mimeType, int sessionId); +/// Reports that the last item provider able to read this session's payload has gone, so the +/// framework may drop it. +/// +/// An NSItemProvider may be kept by a receiver and read long after the gesture ended, so the +/// payload cannot be released on any schedule of the drag's own -- holding a fixed number of +/// recent drags instead means a provider older than that answers nothing. The providers +/// themselves say when they are done: each one retains a token for the session, and the last +/// release calls this. +void CN1NativeDragDeliverPayloadReleased(int sessionId); + /// Announces that UIKit has started a drag session. Returns the actions the framework's staged /// operation allows, or 0 when it has none -- in which case no drag begins. The Java side calls /// CN1BeginNativeDragPayload and CN1AddNativeDragPayload from inside this call. diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index df672386cd9..aa3a99a2ba9 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -439,6 +439,37 @@ void CN1CancelNativeDrag(void) { cn1SessionActions = CN1_DND_ACTION_NONE; } +/// Holds a drag's payload alive for exactly as long as something can still read it. +/// +/// Every load handler this session registers captures the token, and copying a block retains +/// what it captures, so the token outlives the gesture precisely when an NSItemProvider does -- +/// which is the case that matters, since a receiver may keep a provider and read it much later. +/// When the last provider goes the token deallocates and the framework drops the payload. It +/// carries only the session id: the bytes stay on the Java side, unbuilt until read. +@interface CN1DragPayloadToken : NSObject { + int _sessionId; +} +@property (nonatomic) int sessionId; +@end + +@implementation CN1DragPayloadToken + +@synthesize sessionId = _sessionId; + +- (void)dealloc { + const int released = _sessionId; + // dealloc runs on whichever thread released the last provider. Every other call into the + // framework from this file is made on the main thread, and this one is no different. + dispatch_async(dispatch_get_main_queue(), ^{ + CN1NativeDragDeliverPayloadReleased(released); + }); +#ifndef CN1_USE_ARC + [super dealloc]; +#endif +} + +@end + API_AVAILABLE(ios(11.0)) @interface CN1DragAndDropDelegate : NSObject @end @@ -458,6 +489,8 @@ @implementation CN1DragAndDropDelegate // receiver reads them. int allowed = CN1NativeDragDeliverSessionStarted(); if (allowed == CN1_DND_ACTION_NONE) { + // Nothing was staged, or nothing may be done with what was. Either way no session + // begins; the framework side has already tidied up whatever it had. return @[]; } // The authoritative set, which is what a local drop session is told the source allows. @@ -468,7 +501,12 @@ @implementation CN1DragAndDropDelegate cn1LocalDropResult = -1; NSMutableArray* items = [NSMutableArray array]; - const int sessionId = cn1DragSessionId; + // The payload's keeper. Released once below, after everything that could read it has been + // registered; from then on the load handlers are its only owners, so it dies with the last + // of them -- and with no handlers at all it dies here, which is equally correct because + // nothing can read the payload then either. + CN1DragPayloadToken* payloadToken = [[CN1DragPayloadToken alloc] init]; + payloadToken.sessionId = cn1DragSessionId; // Every representation the operation declared, registered lazily. The value is fetched when // a receiver reads that type, not now: a drag that is begun and abandoned must not have @@ -487,7 +525,7 @@ @implementation CN1DragAndDropDelegate visibility:NSItemProviderRepresentationVisibilityAll loadHandler:^NSProgress *(void (^completion)(NSData *, NSError *)) { dispatch_async(dispatch_get_main_queue(), ^{ - completion(CN1NativeDragDeliverResolve(mime, sessionId), nil); + completion(CN1NativeDragDeliverResolve(mime, payloadToken.sessionId), nil); }); return nil; }]; @@ -527,6 +565,11 @@ @implementation CN1DragAndDropDelegate [item release]; #endif } +#ifndef CN1_USE_ARC + // The registered load handlers own it from here; under ARC the local strong reference goes + // at the end of this scope and does the same thing. + [payloadToken release]; +#endif if (items.count == 0) { cn1DraggingOut = NO; CN1NativeDragDeliverCompleted(CN1_DND_ACTION_NONE); diff --git a/Ports/iOSPort/nativeSources/IOSNative.m b/Ports/iOSPort/nativeSources/IOSNative.m index 1ef1f637e54..694d8013023 100644 --- a/Ports/iOSPort/nativeSources/IOSNative.m +++ b/Ports/iOSPort/nativeSources/IOSNative.m @@ -1149,6 +1149,11 @@ void CN1NativeDragDeliverDropAddFile(NSString* mimeType, NSString* path) { return bytes == JAVA_NULL ? nil : arrayToData(bytes); } +void CN1NativeDragDeliverPayloadReleased(int sessionId) { + com_codename1_impl_ios_IOSImplementation_nativeDragPayloadReleasedCallback___int( + CN1_THREAD_GET_STATE_PASS_ARG sessionId); +} + int CN1NativeDragDeliverDropCommit(int x, int y, int action) { return (int)com_codename1_impl_ios_IOSImplementation_nativeDropCommitCallback___int_int_int_R_int( CN1_THREAD_GET_STATE_PASS_ARG x, y, action); diff --git a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java index 0c8d8033e74..7f969551189 100644 --- a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java +++ b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java @@ -9271,22 +9271,24 @@ public static void nativeDragExitCallback() { NativeDragAndDrop.dragExit(0); } - /// The two most recent exported drags, each under the session id its load handlers carry. + /// Every exported drag whose payload something can still read, under the session id its + /// load handlers carry. /// /// Deliberately not the active drag: an item provider's load handler is asynchronous by /// design, and a receiving application is free to defer reading a representation until /// after the session has ended -- by which point the active drag has been cleared and the /// lookup would answer with nothing at all. /// - /// Two of them, and matched by id, because a single slot answered a late read with - /// *whatever was being dragged by then*: a handler from the previous drag resolving after - /// the next one began produced that one's bytes. Wrong data is worse than none, so a read - /// older than these answers null instead. Two payloads is the bound: a receiver would have - /// to sit on an unread representation across two further complete drags to fall off it. - private static NativeDragOperation exportedDrag; - private static int exportedDragId; - private static NativeDragOperation previousExportedDrag; - private static int previousExportedDragId; + /// Matched by id, because a single slot answered a late read with *whatever was being + /// dragged by then*: a handler from the previous drag resolving after the next one began + /// produced that one's bytes. Nor is it a fixed number of recent drags -- a receiver may + /// keep an item provider and read it at any later point, and any bound expires payloads + /// that are still legitimately readable. The providers themselves say when they are done: + /// CN1DragAndDrop.m gives every load handler a token for its session and calls + /// #nativeDragPayloadReleasedCallback(int) when the last of them is released, so a payload + /// is held for exactly as long as something can ask for it and no longer. + private static final java.util.Map exportedDrags = + new java.util.HashMap(); private static int nextDragSessionId; /// The drop being assembled by CN1DragAndDrop.m, one representation at a time. @@ -9384,12 +9386,13 @@ public static int nativeDragSessionStartedCallback() { if (op == null) { return 0; } - previousExportedDrag = exportedDrag; - previousExportedDragId = exportedDragId; - exportedDrag = op; - exportedDragId = ++nextDragSessionId; + int sessionId; + synchronized (exportedDrags) { + sessionId = ++nextDragSessionId; + exportedDrags.put(Integer.valueOf(sessionId), op); + } ClipboardContent content = op.getContent(); - nativeInstance.beginNativeDragPayload(exportedDragId); + nativeInstance.beginNativeDragPayload(sessionId); // Names, not values. Reading a representation here would build every promised file and // encode every promised image at the moment the drag begins -- including for a drag the // user then abandons -- which is the opposite of what setDataProvider promises. The @@ -9425,12 +9428,10 @@ public static int nativeDragSessionStartedCallback() { /// its bytes, or null when that drag can no longer supply it public static byte[] nativeDragResolveCallback(String mimeType, int sessionId) { // Matched by session id, not simply the latest: a handler from an earlier drag must - // answer for that drag or not at all. See the fields. - NativeDragOperation op = null; - if (exportedDrag != null && sessionId == exportedDragId) { - op = exportedDrag; - } else if (previousExportedDrag != null && sessionId == previousExportedDragId) { - op = previousExportedDrag; + // answer for that drag or not at all. See the field. + NativeDragOperation op; + synchronized (exportedDrags) { + op = exportedDrags.get(Integer.valueOf(sessionId)); } if (op == null || mimeType == null || mimeType.length() == 0) { return null; @@ -9464,6 +9465,18 @@ public static void nativeDragCompletedCallback(int action) { NativeDragAndDrop.dragCompleted(action); } + /// Invoked from CN1DragAndDrop.m once the last item provider that could read a drag's + /// payload has been released, which is the only moment the payload is certainly unreadable. + /// + /// #### Parameters + /// + /// - `sessionId`: the drag whose payload may be dropped + public static void nativeDragPayloadReleasedCallback(int sessionId) { + synchronized (exportedDrags) { + exportedDrags.remove(Integer.valueOf(sessionId)); + } + } + @Override public void copyToClipboard(Object obj) { if(obj instanceof com.codename1.ui.ClipboardContent) { diff --git a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java index 8146493a023..7c1a17059d1 100644 --- a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java +++ b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java @@ -782,6 +782,80 @@ void aPlatformThatOwnsTheGestureKeepsTheStagedOperationUntilItStarts() { } } + @FormTest + void anOperationThatPermitsNothingNeverBecomesADrag() { + implementation.resetNativeDragState(); + implementation.setNativeDragAndDropSupported(true); + try { + Form form = Display.getInstance().getCurrent(); + Container source = new Container(); + NativeDragOperation op = new NativeDragOperation("nothing may be done with me") + .setAllowedActions(NativeDragOperation.ACTION_NONE); + source.setNativeDragOperation(op); + form.setLayout(new BorderLayout()); + form.add(BorderLayout.CENTER, source); + form.revalidate(); + + int x = source.getAbsoluteX() + 10; + int y = source.getAbsoluteY() + 10; + form.pointerPressed(x, y); + form.pointerDragged(x + 200, y + 200); + assertNull(implementation.getStartedNativeDrag(), + "no receiver could accept it, so there is no drag to run"); + assertFalse(NativeDragAndDrop.startDrag(source, op), + "and asking for one directly is refused on the same terms"); + assertNull(NativeDragAndDrop.dragSessionStarted(), + "a platform whose own recognizer fires later gets the same answer -- a " + + "session that can never complete would wedge every drag after it"); + assertNull(NativeDragAndDrop.getActiveDrag()); + + form.pointerReleased(x + 200, y + 200); + } finally { + implementation.setNativeDragAndDropSupported(false); + implementation.resetNativeDragState(); + } + } + + @FormTest + void aDeferredSessionGivesTheSourceBackTheVisibilityTheLightweightDragTook() { + implementation.resetNativeDragState(); + implementation.setNativeDragAndDropSupported(true); + // The port refuses the start, so the gesture carries on as a lightweight drag -- which + // hides the source and carries its image -- until the platform's own recognizer fires. + implementation.setNativeDragStartRefused(true); + try { + Form form = Display.getInstance().getCurrent(); + Container source = new Container(); + source.setDraggable(true); + NativeDragOperation op = new NativeDragOperation("dragged out"); + source.setNativeDragOperation(op); + form.setLayout(new BorderLayout()); + form.add(BorderLayout.CENTER, source); + form.revalidate(); + + int x = source.getAbsoluteX() + 10; + int y = source.getAbsoluteY() + 10; + form.pointerPressed(x, y); + form.pointerDragged(x + 200, y + 200); + assertFalse(source.isVisible(), + "the lightweight drag took the gesture and hid the source it is carrying"); + + assertSame(op, NativeDragAndDrop.dragSessionStarted()); + flushSerialCalls(); + assertTrue(source.isVisible(), + "the native session draws its own preview and never runs the lightweight " + + "drop, so nothing else would ever make the source visible again"); + + NativeDragAndDrop.dragCompleted(NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + } finally { + implementation.setNativeDragStartRefused(false); + implementation.setNativeDragAndDropSupported(false); + implementation.resetNativeDragState(); + NativeDragAndDrop.dragCompleted(NativeDragOperation.ACTION_NONE); + } + } + @FormTest void aReleaseAfterARefusedStartDoesNotLeaveTheDragArmed() { implementation.resetNativeDragState(); diff --git a/maven/javase/src/test/java/com/codename1/impl/javase/JavaSENativeDragAndDropTest.java b/maven/javase/src/test/java/com/codename1/impl/javase/JavaSENativeDragAndDropTest.java index 0a9290dc68f..e19a16d915c 100644 --- a/maven/javase/src/test/java/com/codename1/impl/javase/JavaSENativeDragAndDropTest.java +++ b/maven/javase/src/test/java/com/codename1/impl/javase/JavaSENativeDragAndDropTest.java @@ -177,6 +177,54 @@ void binaryContentIsOfferedAsAStream() throws Exception { assertArrayEquals(pdf, read); } + @Test + void anImageEncodingNothingCanDecodeDoesNotClaimTheStandardImageFlavor() { + ClipboardContent content = new ClipboardContent() + .setData("image/webp", new byte[]{'R', 'I', 'F', 'F'}); + Transferable t = new JavaSEPort.RichTransferable(content); + + assertFalse(t.isDataFlavorSupported(DataFlavor.imageFlavor), + "a desktop receiver commonly picks the standard image flavor ahead of the " + + "MIME specific stream, so claiming it for an encoding ImageIO cannot " + + "read loses the whole drop to an UnsupportedFlavorException"); + assertNotNull(flavorFor(t, "image/webp"), + "the bytes are still perfectly readable by anything that wants that type"); + } + + @Test + void aDecodableImageStillClaimsTheStandardImageFlavor() throws Exception { + ClipboardContent content = new ClipboardContent() + .setData(ClipboardContent.MIME_PNG, onePixelPng()); + Transferable t = new JavaSEPort.RichTransferable(content); + + assertTrue(t.isDataFlavorSupported(DataFlavor.imageFlavor)); + assertTrue(t.getTransferData(DataFlavor.imageFlavor) instanceof java.awt.Image); + } + + @Test + void theStandardImageFlavorServesAnyEncodingItWasAdvertisedFor() throws Exception { + java.io.ByteArrayOutputStream bmp = new java.io.ByteArrayOutputStream(); + javax.imageio.ImageIO.write( + new java.awt.image.BufferedImage(1, 1, java.awt.image.BufferedImage.TYPE_INT_RGB), + "bmp", bmp); + ClipboardContent content = new ClipboardContent().setData("image/bmp", bmp.toByteArray()); + Transferable t = new JavaSEPort.RichTransferable(content); + + assertTrue(t.isDataFlavorSupported(DataFlavor.imageFlavor), + "ImageIO reads BMP, so the flavor is advertised"); + assertTrue(t.getTransferData(DataFlavor.imageFlavor) instanceof java.awt.Image, + "and what is advertised has to be servable -- reading only the three encodings " + + "the framework names left this one throwing"); + } + + private static byte[] onePixelPng() throws IOException { + java.io.ByteArrayOutputStream out = new java.io.ByteArrayOutputStream(); + javax.imageio.ImageIO.write( + new java.awt.image.BufferedImage(1, 1, java.awt.image.BufferedImage.TYPE_INT_ARGB), + "png", out); + return out.toByteArray(); + } + @Test void anUnofferedFlavorIsRefusedRatherThanAnsweredWithNull() { ClipboardContent content = new ClipboardContent().setData(ClipboardContent.MIME_TEXT, "hi"); From 2452f1f542f6076d733db8d86343aa19ba0e30b1 Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 19:36:00 +0300 Subject: [PATCH 16/26] Review: empty markup is a value, and two clip files could be one file **Empty HTML was read as no HTML.** ClipData.Item.getHtmlText answers null when the item carries no markup at all, so anything else -- "" included -- is what the source published. Discarding it left fillAdvertisedTypes to rebuild the advertised text/html from the plain text, and the target got something the source never wrote. This exporter publishes exactly that item for content whose HTML is empty, so it is our own round trip that loses it. The plain-text branch beside it keeps its length test, and that asymmetry is deliberate: coerceToText *derives* text from whatever the item holds, so an empty answer there means the item had nothing to give rather than that the source published nothing -- and accepting it would stop the search before an item that does carry the text. **Two representations could end up sharing a file.** The generated name was the clock plus the payload's length, so two representations of one payload that share an extension and a byte length -- written in the same millisecond, which is what a loop does -- produced the same path. The second write overwrote the first and both clip items then pointed at the second one's bytes. createTempFile is the guarantee rather than a longer guess, and it keeps the extension, which is what names the type an untyped URI carries when it is read back. Co-Authored-By: Claude Opus 5 (1M context) --- .../impl/android/AndroidImplementation.java | 25 +++++++++++++------ 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java index 905cd37faac..da037a35b7c 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java @@ -10391,9 +10391,14 @@ private Uri writeAsProviderUri(byte[] bytes, String extension) throws IOExceptio // A zero length payload is still a payload: refusing it would leave the clip without a // type it had advertised, and a target filtering on that type would accept the hover // and be refused the drop. - File file = new File(new File(getContext().getCacheDir(), "intent_files"), - "cn1-clip-" + System.currentTimeMillis() + "-" + bytes.length + "." + extension); - file.getParentFile().mkdirs(); + File dir = new File(getContext().getCacheDir(), "intent_files"); + dir.mkdirs(); + // A name built from the clock and the payload's length collided: two representations of + // one payload that share an extension and a byte length are written within the same + // millisecond, and the second overwrote the first -- leaving both clip items pointing at + // the second one's bytes. createTempFile is the guarantee rather than a longer guess, + // and it keeps the extension, which is what names the type this URI carries. + File file = File.createTempFile("cn1-clip-", "." + extension, dir); OutputStream os = new FileOutputStream(file); try { os.write(bytes); @@ -10599,12 +10604,18 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { com.codename1.io.Log.e(t); } if (html == null && sdk >= 16) { - String itemHtml = item.getHtmlText(); - if (itemHtml != null && itemHtml.length() > 0) { - html = itemHtml; - } + // Empty markup is a value, not an absence: getHtmlText answers null when the + // item carries no HTML at all, so anything else is what the source published. + // Discarding it left fillAdvertisedTypes to rebuild the advertised text/html + // from the plain text, handing the target something the source never wrote -- + // and this exporter publishes exactly that item for content whose HTML is empty. + html = item.getHtmlText(); } if (plain == null) { + // Not the same test. coerceToText *derives* text from whatever the item holds, + // so an empty answer means it had nothing to give rather than that the source + // published nothing -- and accepting it would stop the search before an item + // that does carry the text. CharSequence text = item.coerceToText(getContext()); if (text != null && text.length() > 0) { plain = text.toString(); From f67663b4cb0a8636028f80155ef38d269bbad80e Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 20:08:37 +0300 Subject: [PATCH 17/26] Review: an image that stopped being a file, an extension that could not tell two types apart, and text stored as bytes **A dragged image file arrived without being a file.** MIME_FILE paths go out as FileProvider URIs and the description advertises text/uri-list, but the read gave an image/* URI to the decode branch and returned before recording it. Drag a PNG *file* and the drop carried image bytes and no file at all, so a target filtering on MIME_FILE accepted the hover and was refused. Every URI item is a file reference as well as whatever its type made of it, and it is recorded as one now. The image read also gets its own catch, because bytes are one representation of a URI rather than the whole of it and a failed read must not take the file reference with it. **Two types could produce URIs nothing could tell apart.** Recovering the type from the extension was a lossy inversion: extensionForMime strips at the plus, so application/x-foo and application/x-foo+json both reduce to xfoo, and two representations of one payload then matched each other's URIs, were both left unassigned, and were both dropped. So the name carries the type outright now, hex encoded -- unlovely for a file nobody reads, and exact, where the extension can only ever be a guess. The guess remains for a clip this application did not write, which is the only thing it was ever answering. **A text flavor carried as bytes was stored as bytes.** text/html;class="[B" and text/plain;class=java.io.InputStream are both ordinary on the desktop, and the encoding is a parameter of the flavor rather than something to assume. Storing the bytes made getText() answer null for a type the drop had just accepted, and decoding them as UTF-8 by hand gets a charset=UTF-16 flavor wrong -- so they go through the reader the flavor itself supplies. A String representation still short circuits, so the ordinary flavor is not read twice. Both text-flavor tests fail with the branch disabled. Co-Authored-By: Claude Opus 5 (1M context) --- .../impl/android/AndroidImplementation.java | 133 ++++++++++++++---- .../impl/javase/JavaSENativeDragAndDrop.java | 37 +++++ .../javase/JavaSENativeDragAndDropTest.java | 24 ++++ 3 files changed, 166 insertions(+), 28 deletions(-) diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java index da037a35b7c..397f5abadfe 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java @@ -10296,7 +10296,7 @@ private void addBinaryContent(ClipboardContent content, List mimeTypes, imageExt = "gif"; } if (imageBytes != null) { - Uri imageUri = writeAsProviderUri(imageBytes, imageExt); + Uri imageUri = writeAsProviderUri(imageBytes, imageExt, imageMime); if (imageUri != null) { if (!mimeTypes.contains(imageMime)) { mimeTypes.add(imageMime); @@ -10370,7 +10370,7 @@ private void addRemainingRepresentations(ClipboardContent content, String carrie bytes = (byte[]) value; } if (bytes != null) { - Uri uri = writeAsProviderUri(bytes, extensionForMime(mime)); + Uri uri = writeAsProviderUri(bytes, extensionForMime(mime), mime); if (uri != null) { mimeTypes.add(mime); items.add(new ClipData.Item(uri)); @@ -10384,7 +10384,10 @@ private void addRemainingRepresentations(ClipboardContent content, String carrie /// /// AndroidGradleBuilder exposes cache/intent_files through the app's FileProvider, so /// generated payloads stay inside that root and FileProvider can safely name them. - private Uri writeAsProviderUri(byte[] bytes, String extension) throws IOException { + /// + /// The name carries `mime` so the read back is an answer rather than a guess -- see + /// `#decodeMimeFromFileName(java.lang.String)`. + private Uri writeAsProviderUri(byte[] bytes, String extension, String mime) throws IOException { if (bytes == null) { return null; } @@ -10396,9 +10399,11 @@ private Uri writeAsProviderUri(byte[] bytes, String extension) throws IOExceptio // A name built from the clock and the payload's length collided: two representations of // one payload that share an extension and a byte length are written within the same // millisecond, and the second overwrote the first -- leaving both clip items pointing at - // the second one's bytes. createTempFile is the guarantee rather than a longer guess, - // and it keeps the extension, which is what names the type this URI carries. - File file = File.createTempFile("cn1-clip-", "." + extension, dir); + // the second one's bytes. createTempFile is the guarantee rather than a longer guess. + String encoded = encodeMimeForFileName(mime); + File file = File.createTempFile( + encoded == null ? CLIP_FILE_PREFIX : CLIP_FILE_PREFIX + encoded + "-", + "." + extension, dir); OutputStream os = new FileOutputStream(file); try { os.write(bytes); @@ -10412,6 +10417,63 @@ private Uri writeAsProviderUri(byte[] bytes, String extension) throws IOExceptio return uri; } + /// The name every generated clip file starts with, and the alphabet + /// `#encodeMimeForFileName(java.lang.String)` writes the type in. + private static final String CLIP_FILE_PREFIX = "cn1-clip-"; + private static final String CLIP_MIME_HEX = "0123456789abcdef"; + + /// Writes a MIME type into something that is legal in a file name and reads back as itself. + /// + /// The extension cannot do this job. It is derived from the type and the derivation is + /// lossy -- `application/x-foo` and `application/x-foo+json` both reduce to `xfoo` -- so two + /// representations of one payload can produce URIs no reader can tell apart, and both are + /// then dropped rather than mispaired. Hex is unlovely for a file name nobody reads, and it + /// is exact: every byte of the type survives, and no character it produces means anything to + /// a file system, a URI or `#decodeMimeFromFileName(java.lang.String)`. + /// + /// Answers null for a type this cannot carry, and the file is then named without one. + private static String encodeMimeForFileName(String mime) { + if (mime == null || mime.length() == 0 || mime.length() > 60) { + return null; + } + StringBuilder out = new StringBuilder(); + for (int iter = 0; iter < mime.length(); iter++) { + int c = mime.charAt(iter); + if (c > 0xff) { + return null; + } + out.append(CLIP_MIME_HEX.charAt((c >> 4) & 0xf)).append(CLIP_MIME_HEX.charAt(c & 0xf)); + } + return out.toString(); + } + + /// The MIME type `#encodeMimeForFileName(java.lang.String)` wrote into this name, or null + /// when the name did not come from there -- a clip another application published, or one + /// whose type was too long to carry. + private static String decodeMimeFromFileName(String name) { + if (name == null || !name.startsWith(CLIP_FILE_PREFIX)) { + return null; + } + int end = name.indexOf('-', CLIP_FILE_PREFIX.length()); + if (end < 0) { + return null; + } + String hex = name.substring(CLIP_FILE_PREFIX.length(), end); + if (hex.length() == 0 || (hex.length() & 1) != 0) { + return null; + } + StringBuilder out = new StringBuilder(); + for (int iter = 0; iter < hex.length(); iter += 2) { + int hi = Character.digit(hex.charAt(iter), 16); + int lo = Character.digit(hex.charAt(iter + 1), 16); + if (hi < 0 || lo < 0) { + return null; + } + out.append((char) ((hi << 4) | lo)); + } + return out.toString().toLowerCase(); + } + /// A file extension for a MIME type, used to name the temporary file a content URI is /// served from. /// @@ -10572,31 +10634,40 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { if (uri != null) { String type = getContext().getContentResolver().getType(uri); if (type != null && type.startsWith("image/")) { - InputStream in = getContext().getContentResolver().openInputStream(uri); - if (in != null) { - try { - byte[] bytes = Util.readInputStream(in); - content.setData(imageMimeFor(type), bytes); - } finally { - in.close(); + try { + InputStream in = getContext().getContentResolver().openInputStream(uri); + if (in != null) { + try { + byte[] bytes = Util.readInputStream(in); + content.setData(imageMimeFor(type), bytes); + } finally { + in.close(); + } } + } catch (Throwable t) { + // The bytes are one representation of this URI, not the whole of + // it. A read that fails must still leave the file reference below. + com.codename1.io.Log.e(t); } - continue; - } - // A typed URI is a file reference *and* that type. Reducing it to a file - // alone let a target filtering on, say, application/pdf accept the hover -- - // the description advertised the type -- and then be refused the drop, - // because the content it is filtered against a second time no longer had - // it. The bytes are promised rather than read: a target that only wants the - // path should not pay for a document it never opens. - if (type != null && type.length() > 0 + } else if (type != null && type.length() > 0 && !"application/octet-stream".equals(type.toLowerCase())) { + // A typed URI is a file reference *and* that type. Reducing it to a file + // alone let a target filtering on, say, application/pdf accept the hover + // -- the description advertised the type -- and then be refused the + // drop, because the content it is filtered against a second time no + // longer had it. The bytes are promised rather than read: a target that + // only wants the path should not pay for a document it never opens. if (!content.hasMimeType(type)) { content.setDataProvider(type.toLowerCase(), uriBytesProvider(uri)); } } else { unnamedUris.add(uri); } + // Every URI item is a file reference as well as whatever its type made of + // it. The image branch returned before reaching this, so dragging a PNG + // *file* produced image bytes and no file at all -- and a target filtering + // on MIME_FILE accepted the hover, because the description still advertised + // text/uri-list, and was then refused the drop. fileUris.add(uri.toString()); continue; } @@ -10755,17 +10826,23 @@ private void fillAdvertisedTypes(ClipboardContent content, ClipDescription descr /// serves. /// /// ContentResolver could not name it -- MimeTypeMap has no entry for an application defined - /// type, so the FileProvider serving it reports octet-stream -- but the extension is still - /// exactly what `#extensionForMime(java.lang.String)` produced for the type that was - /// written, so the association the resolver lost is recoverable rather than guessed. That is - /// what lets more than one unnameable representation survive the round trip. An extension - /// two advertised types share answers nothing, as does a clip this application did not - /// write. + /// type, so the FileProvider serving it reports octet-stream. What this application wrote + /// still says so in its own name, exactly, which is the answer; a clip from elsewhere gets + /// the extension read as a type, which is a good guess and is treated as one -- an extension + /// two advertised types share answers nothing. private String mimeForUnnamedUri(Uri uri, List binary, List text) { String name = displayNameFor(uri); if (name == null) { return null; } + String declared = decodeMimeFromFileName(name); + if (declared != null) { + // Written by this application, which named the type outright. It answers even when + // it names a type that is not among the candidates -- that means the type is already + // satisfied, or was never advertised, and either way this URI is not the missing + // one. Guessing past an exact answer would be strictly worse. + return binary.contains(declared) || text.contains(declared) ? declared : null; + } int dot = name.lastIndexOf('.'); if (dot < 0 || dot == name.length() - 1) { return null; diff --git a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java index 65a029d9de1..aba955f3af1 100644 --- a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java +++ b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java @@ -407,6 +407,18 @@ private static Object readValue(Transferable transferable, DataFlavor flavor, St } return null; } + if (mime.startsWith("text/") && !(out instanceof String) && flavor.isFlavorTextType()) { + // A text flavor is free to hand over bytes -- text/html;class="[B" and + // text/plain;class=java.io.InputStream are both ordinary on the desktop -- and + // the encoding those bytes are in is a parameter of the flavor, not something + // to assume. Storing them as a binary payload made getText() answer null for a + // type the drop had just accepted, and decoding them as UTF-8 by hand would get + // a charset=UTF-16 flavor wrong. DataFlavor's own reader is what knows. + String text = textFromFlavor(transferable, flavor); + if (text != null) { + return text; + } + } if (out instanceof byte[]) { return out; } @@ -421,6 +433,31 @@ private static Object readValue(Transferable transferable, DataFlavor flavor, St } } + /// Reads a text flavor through the reader the flavor itself supplies, which applies the + /// charset the flavor declares. Returns null when the flavor will not produce one, leaving + /// the caller's own handling to run. + private static String textFromFlavor(Transferable transferable, DataFlavor flavor) { + try { + java.io.Reader reader = flavor.getReaderForText(transferable); + if (reader == null) { + return null; + } + try { + StringBuilder out = new StringBuilder(); + char[] buffer = new char[2048]; + int read; + while ((read = reader.read(buffer)) >= 0) { + out.append(buffer, 0, read); + } + return out.toString(); + } finally { + reader.close(); + } + } catch (Throwable err) { + return null; + } + } + /// Turns whatever a file flavor produced -- a list of files, or a URI list as text -- into /// absolute paths. private static Object filePaths(Object value) throws Exception { diff --git a/maven/javase/src/test/java/com/codename1/impl/javase/JavaSENativeDragAndDropTest.java b/maven/javase/src/test/java/com/codename1/impl/javase/JavaSENativeDragAndDropTest.java index e19a16d915c..a3698970e86 100644 --- a/maven/javase/src/test/java/com/codename1/impl/javase/JavaSENativeDragAndDropTest.java +++ b/maven/javase/src/test/java/com/codename1/impl/javase/JavaSENativeDragAndDropTest.java @@ -251,6 +251,30 @@ void describingADragInProgressReadsNoData() { + "platforms the data does not exist until the drop"); } + @Test + void aTextFlavorCarriedAsBytesStillReadsAsText() throws Exception { + DataFlavor htmlBytes = new DataFlavor("text/html;charset=UTF-8;class=\"[B\""); + FakeTransferable t = new FakeTransferable() + .add(DataFlavor.stringFlavor, "plain") + .add(htmlBytes, "hi".getBytes("UTF-8")); + + ClipboardContent content = JavaSENativeDragAndDrop.contentFor(t, t.getTransferDataFlavors(), true); + assertEquals("hi", content.getText(ClipboardContent.MIME_HTML), + "the flavor declares its own charset; storing its bytes as a binary payload " + + "made getText() null for a type the drop had just accepted"); + } + + @Test + void aTextFlavorCarriedAsAStreamStillReadsAsText() throws Exception { + DataFlavor htmlStream = new DataFlavor("text/html;charset=UTF-16;class=java.io.InputStream"); + FakeTransferable t = new FakeTransferable() + .add(htmlStream, new ByteArrayInputStream("x".getBytes("UTF-16"))); + + ClipboardContent content = JavaSENativeDragAndDrop.contentFor(t, t.getTransferDataFlavors(), true); + assertEquals("x", content.getText(ClipboardContent.MIME_HTML), + "and a charset that is not UTF-8 has to be honoured rather than assumed"); + } + @Test void aDroppedFileListBecomesFilePaths() { FakeTransferable t = new FakeTransferable() From db0c0c034f82396946e8510cf300fa1312dec5c0 Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 20:39:50 +0300 Subject: [PATCH 18/26] Review: a dragged document arriving empty, and a fallback served the file's bytes **A dragged .txt arrived empty.** A URI the resolver types text/plain registers the document's own contents under MIME_TEXT, and the unconditional write at the end of the read -- there so that every clip reports some text -- then replaced that provider with the empty string. It is synthesized now only when nothing else supplied the text. The resolver's answer is also reduced to a bare lower case MIME type first, because that guard compares against text/plain and a provider answering "text/plain; charset=utf-8" would both file the document under a type no target asks for and slip straight past it. **A file's alternatives were served the file's bytes.** The outgoing side deliberately attaches an operation's other representations to the file's own NSItemProvider -- adding a second item made UIKit expose them as two dragged things, so a receiver could import the document *and* a stray piece of text instead of choosing the best form of one. The reading side then saw public.file-url, skipped every one of those handlers, and named the copied document under all of their types, so this framework's own file-with-a-text- fallback drag delivered the file body under the fallback's type. They are loaded now -- as files rather than as data, which is what keeps the reason the shortcut existed. Reading a representation that really is the document into memory on top of copying it is how an application runs out of memory, so the loads are issued from inside the file handler where the document's own path is known: an alternative that resolves to that path is another name for the document and shares its single copy, and one that resolves elsewhere is a representation of its own and gets a copy of its own. One case this cannot resolve is recorded where the source registers them: a declared type whose UTI is also the file's own -- a text/plain fallback beside a .txt -- registers a second representation under an identifier the provider already vends, and which of the two a receiver gets is NSItemProvider's choice. Both are honestly that type, so neither answer is wrong and there is no way to say which was meant. Co-Authored-By: Claude Opus 5 (1M context) --- .../impl/android/AndroidImplementation.java | 35 +++++++- Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 89 ++++++++++++++----- 2 files changed, 98 insertions(+), 26 deletions(-) diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java index 397f5abadfe..7d78e7c35d0 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java @@ -10632,7 +10632,12 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { try { Uri uri = item.getUri(); if (uri != null) { - String type = getContext().getContentResolver().getType(uri); + // Without the parameters, because a bare MIME type is what everything here + // compares against: a provider answering "text/plain; charset=utf-8" would + // file the document under a type no target asks for, and would slip past + // the MIME_TEXT check below that stops the synthesized empty text from + // overwriting it. + String type = bareMimeType(getContext().getContentResolver().getType(uri)); if (type != null && type.startsWith("image/")) { try { InputStream in = getContext().getContentResolver().openInputStream(uri); @@ -10650,7 +10655,7 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { com.codename1.io.Log.e(t); } } else if (type != null && type.length() > 0 - && !"application/octet-stream".equals(type.toLowerCase())) { + && !"application/octet-stream".equals(type)) { // A typed URI is a file reference *and* that type. Reducing it to a file // alone let a target filtering on, say, application/pdf accept the hover // -- the description advertised the type -- and then be refused the @@ -10658,7 +10663,7 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { // longer had it. The bytes are promised rather than read: a target that // only wants the path should not pay for a document it never opens. if (!content.hasMimeType(type)) { - content.setDataProvider(type.toLowerCase(), uriBytesProvider(uri)); + content.setDataProvider(type, uriBytesProvider(uri)); } } else { unnamedUris.add(uri); @@ -10694,18 +10699,40 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { } } if (html != null) { + // A value the clip's own item published, so it wins over a URI the resolver happened + // to type text/html -- an .html file being dragged. Same rule as the text below, + // and the reason that one needs a guard and this one does not: there is no + // synthesized empty HTML to write over a representation that already answered. content.setData(ClipboardContent.MIME_HTML, html); } if (!fileUris.isEmpty()) { content.setFiles(fileUris.toArray(new String[fileUris.size()])); } - content.setData(ClipboardContent.MIME_TEXT, plain == null ? "" : plain); + if (plain != null) { + content.setData(ClipboardContent.MIME_TEXT, plain); + } else if (!content.hasMimeType(ClipboardContent.MIME_TEXT)) { + // Every clip reports text, so a target asking for it gets "" rather than nothing -- + // but only when nothing else has supplied it. A URI the resolver typed text/plain, + // which is what a dragged .txt is, has already registered the document's own + // contents, and writing over that handed the target an empty document. + content.setData(ClipboardContent.MIME_TEXT, ""); + } if (description != null) { fillAdvertisedTypes(content, description, plain, fileUris, unnamedUris); } return content; } + /// A MIME type without its parameters, lower case, or null when there is none. + private static String bareMimeType(String type) { + if (type == null) { + return null; + } + int semicolon = type.indexOf(';'); + String bare = (semicolon < 0 ? type : type.substring(0, semicolon)).trim().toLowerCase(); + return bare.length() == 0 ? null : bare; + } + /// Reads a content URI's bytes when something actually asks for them. /// /// The drag-and-drop permission this drop was granted lasts for the life of the activity -- diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index aa3a99a2ba9..66bf5804a36 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -470,6 +470,29 @@ - (void)dealloc { @end +/// Copies a file a drop handed over into somewhere that outlives the handler, and returns the +/// path -- or nil when the copy failed. +/// +/// Every URL an NSItemProvider produces is valid only for the duration of the completion handler +/// it arrives in, so a path handed to the framework without copying is unreadable by the time +/// the event dispatch thread sees it. The name is kept because a receiver commonly shows it. +static NSString* cn1CopyDroppedFile(NSURL* url) { + NSString* name = url.lastPathComponent; + if (name == nil || name.length == 0) { + name = @"dropped"; + } + NSString* target = [NSTemporaryDirectory() stringByAppendingPathComponent: + [NSString stringWithFormat:@"cn1-drop-%@-%@", + [[NSUUID UUID] UUIDString], name]]; + NSError* copyError = nil; + if (![[NSFileManager defaultManager] copyItemAtURL:url + toURL:[NSURL fileURLWithPath:target] + error:©Error]) { + return nil; + } + return target; +} + API_AVAILABLE(ios(11.0)) @interface CN1DragAndDropDelegate : NSObject @end @@ -545,6 +568,13 @@ @implementation CN1DragAndDropDelegate // Given a file and a text fallback, adding a second item made UIKit expose them as // two dragged things, so a receiver could import the document *and* a stray piece // of text instead of choosing the best form of one. + // + // Note the one case this cannot express: a declared type whose UTI is also the + // file's own -- a text/plain fallback beside a .txt -- registers a second + // representation under an identifier the provider already vends, and which of the + // two a receiver gets is NSItemProvider's choice rather than ours. Both are + // honestly that type, so neither answer is wrong; there is simply no way to say + // which was meant. registerDeclared(provider); declaredAttached = YES; } @@ -682,8 +712,10 @@ - (void)dropInteraction:(UIDropInteraction *)interaction performDrop:(id Date: Wed, 2 Sep 2026 21:24:48 +0300 Subject: [PATCH 19/26] Review: a file list that was not a URI list, text decoded as the wrong thing, and why a drop cannot wait **The file/URI-list pair was only presented in one direction.** A drag out of a Linux file manager offers text/uri-list and the read synthesized the file list from it; a drag out of the Finder or Explorer offers javaFileListFlavor and nothing synthesized the URI list, so a component filtered to MIME_URI_LIST refused ordinary file drags from the one source every desktop user has. The inverse is declared now, on demand while the drag hovers and materialized on the drop, which is what this port already publishes on the way out. **Every iOS text representation was decoded as UTF-8.** cn1MimeForUti maps public.utf8-plain-text, public.utf16-plain-text and public.utf16-external-plain-text all onto text/plain, because that is the MIME type they all are, but they do not agree about the bytes. The identifier travels beside the data now and says how to read it. Note why trying UTF-8 first is not a substitute: UTF-16 in little endian without a byte order mark decodes as UTF-8 *successfully*, into text full of NULs, so the fallback never fires and the corruption is silent. Anything the identifier does not pin down goes to NSString's own detection rather than an assumption. **A drop cannot wait for a callback that has not run, and the alternatives are worse.** Review asked drop() to honour a queued nativeDragEnter decision before committing. The race is real -- a drop arriving before that callback runs reads what the target declared rather than what the callback was about to say -- but every way of closing it costs more than it saves, and the reasoning is now in the method rather than only in a review thread nobody reads: - Waiting deadlocks. drop() runs on the native drag thread, which the event dispatch thread blocks on to paint. - Refusing while a callback is outstanding refuses every ordinary drop that lands while the event dispatch thread is a frame behind. - Withholding delivery afterwards is worse than delivering: the platform has already been told the action, and on ACTION_MOVE the source deletes its copy on that word, so the data is destroyed rather than misplaced. What is exact is the declarative refusal. canAcceptNativeDrop and getAcceptedDropActions are consulted by findTarget on the calling thread, on the drop as well as on every drag event, so a target refusing through either is never selected and never receives the drop whatever the event dispatch thread is doing. The class contract now says outright that reject() in a callback is a change of mind honoured from the next event, not a refusal. Both JavaSE fixes have tests; both fail with the new branch disabled. Co-Authored-By: Claude Opus 5 (1M context) --- .../com/codename1/ui/NativeDragAndDrop.java | 27 ++++++++- .../impl/javase/JavaSENativeDragAndDrop.java | 51 +++++++++++++++-- Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 57 +++++++++++++++++-- .../javase/JavaSENativeDragAndDropTest.java | 25 ++++++++ 4 files changed, 150 insertions(+), 10 deletions(-) diff --git a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java index d21b0aaa43e..4016bfa791a 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java @@ -507,7 +507,11 @@ public static int dragEnter(int windowId, int x, int y, ClipboardContent content /// A target that refuses a drop outright should say so through /// `Component#canAcceptNativeDrop(com.codename1.ui.ClipboardContent)` or the accepted MIME /// list instead, both of which are consulted here and are therefore exact from the first - /// event. + /// event -- and from every event, including the drop itself. A `NativeDropEvent#reject()` + /// in a callback is a change of mind rather than a refusal: it is honoured from the next + /// event onward, and a drop landing before the callback has run reads what the target + /// declared. `#drop(int, int, int, com.codename1.ui.ClipboardContent, int)` says why that + /// cannot be closed without doing something worse. /// /// #### Parameters /// @@ -621,6 +625,27 @@ public static int drop(int windowId, int x, int y, ClipboardContent content, int // in a callback that has since run would have had that decision quietly // discarded here, and a refusal turned back into a delivered drop on every // port rather than only the one that was noticed. + // + // "Latest word" is as far as this can go, and deliberately so. A drop that + // arrives before the queued nativeDragEnter has run reads what the target + // *declared* rather than what that callback was about to say, and no + // rearrangement of this method fixes that: + // + // - Waiting for the callback deadlocks. This runs on the native drag thread, + // which the event dispatch thread blocks on to paint. + // - Refusing whenever a callback is outstanding refuses every ordinary drop + // that lands while the event dispatch thread is a frame behind. + // - Withholding delivery afterwards is worse than delivering. The platform has + // already been told the action; on ACTION_MOVE the source deletes its copy on + // that word, so a drop withheld after the fact destroys the data instead of + // misplacing it. + // + // What is exact is the declarative refusal: canAcceptNativeDrop and + // getAcceptedDropActions are consulted by findTarget on this thread, here as + // well as on every drag event, so a target refusing through either is never + // selected and never receives the drop, whatever the event dispatch thread is + // doing. That is what a target refusing outright must use -- reject() in a + // callback is a late change of mind, honoured from the next event onward. accepted = currentAction; } else { // A different component from the one the callbacks were about: the pointer diff --git a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java index aba955f3af1..1d7ff99b843 100644 --- a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java +++ b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java @@ -356,10 +356,15 @@ public Object getClipboardData(String requested) { } } // A file list is also a URI list as far as most applications are concerned, and a drag - // out of a Linux file manager offers only the latter. Presenting both means a drop - // target that asks for files gets them either way. Declared the same way the rest of - // the content is -- eagerly on a drop, on demand during a drag -- so describing a drag - // still reads nothing. + // out of a Linux file manager offers only the latter while one out of the Finder offers + // only the former. Presenting both means a drop target that asks for either gets them + // whichever spelling the source used -- the same pair this port publishes on the way + // out. Declared the same way the rest of the content is -- eagerly on a drop, on demand + // during a drag -- so describing a drag still reads nothing. + // + // Only one of the two can fire for any one content, since each is conditioned on the + // other spelling being the one that is present, so neither provider can end up reading + // the other. if (!content.hasMimeType(ClipboardContent.MIME_FILE) && content.hasMimeType(ClipboardContent.MIME_URI_LIST)) { if (eager) { content.setFiles(pathsFromUriList(content.getText(ClipboardContent.MIME_URI_LIST))); @@ -376,10 +381,48 @@ public Object getClipboardData(String requested) { } }); } + } else if (!content.hasMimeType(ClipboardContent.MIME_URI_LIST) + && content.hasMimeType(ClipboardContent.MIME_FILE)) { + // The other direction, which was missing: a Finder or Explorer drag offers only + // javaFileListFlavor, so a component filtered to MIME_URI_LIST refused an ordinary + // file drag from the one source every desktop user has. + if (eager) { + String uris = uriListFrom(content); + if (uris != null) { + content.setData(ClipboardContent.MIME_URI_LIST, uris); + } + } else { + final ClipboardContent describing = content; + content.setDataProvider(ClipboardContent.MIME_URI_LIST, new ClipboardDataProvider() { + @Override + public Object getClipboardData(String requested) { + return uriListFrom(describing); + } + }); + } } return content; } + /// The `text/uri-list` spelling of a content's files: one `file:` URI per line, CRLF + /// separated as RFC 2483 has it, or null when it names none. + private static String uriListFrom(ClipboardContent content) { + String[] paths = content.getFiles(); + if (paths == null || paths.length == 0) { + return null; + } + StringBuilder out = new StringBuilder(); + for (int iter = 0; iter < paths.length; iter++) { + String path = paths[iter]; + if (path == null || path.length() == 0) { + continue; + } + out.append(path.startsWith("file:") ? path : new File(path).toURI().toString()); + out.append("\r\n"); + } + return out.length() == 0 ? null : out.toString(); + } + /// Reads one representation out of a transferable, converting it into the value type the /// MIME type implies. Returns null rather than throwing: a flavor that turns out to be /// unreadable is simply one the drop does not offer. diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index 66bf5804a36..96182832591 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -470,6 +470,48 @@ - (void)dealloc { @end +/// Reads a text representation with the encoding its uniform type identifier declares. +/// +/// cn1MimeForUti maps public.utf8-plain-text, public.utf16-plain-text and +/// public.utf16-external-plain-text all onto text/plain, because that is the MIME type they all +/// are -- but they do not agree about the bytes. Decoding UTF-16 as UTF-8 answers nil, which +/// dropped a representation the drag had advertised and refused the very target that accepted +/// it on the strength of it. Worse, UTF-16 in little endian without a byte order mark decodes +/// as UTF-8 *successfully*, into text full of NULs, so trying UTF-8 first and falling back is +/// not a substitute for reading what the identifier says. +/// +/// Anything the identifier does not pin down is handed to NSString's own detection rather than +/// assumed, and only a representation nothing can read at all comes back nil. +static NSString* cn1TextFromData(NSData* data, NSString* uti) { + NSStringEncoding declared = 0; + if ([uti isEqualToString:@"public.utf16-plain-text"] + || [uti isEqualToString:@"public.utf16-external-plain-text"]) { + declared = NSUTF16StringEncoding; + } else if ([uti isEqualToString:@"public.utf8-plain-text"]) { + declared = NSUTF8StringEncoding; + } + if (declared != 0) { + NSString* exact = [[[NSString alloc] initWithData:data encoding:declared] autorelease]; + if (exact != nil) { + return exact; + } + } + NSString* utf8 = [[[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding] autorelease]; + if (utf8 != nil) { + return utf8; + } + NSString* detected = nil; + [NSString stringEncodingForData:data + encodingOptions:@{NSStringEncodingDetectionSuggestedEncodingsKey: + @[@(NSUTF16StringEncoding), + @(NSUTF16LittleEndianStringEncoding), + @(NSUTF16BigEndianStringEncoding), + @(NSISOLatin1StringEncoding)]} + convertedString:&detected + usedLossyConversion:NULL]; + return detected; +} + /// Copies a file a drop handed over into somewhere that outlives the handler, and returns the /// path -- or nil when the copy failed. /// @@ -710,6 +752,9 @@ - (void)dropInteraction:(UIDropInteraction *)interaction performDrop:(id {data, the uniform type identifier it arrived under}. The identifier is kept + // because it is what says how to read the bytes; see cn1TextFromData. NSMutableDictionary* collected = [[NSMutableDictionary alloc] init]; NSMutableArray* files = [[NSMutableArray alloc] init]; // The representations a file-vending provider also advertises, each named against a file on @@ -795,7 +840,10 @@ - (void)dropInteraction:(UIDropInteraction *)interaction performDrop:(id Date: Wed, 2 Sep 2026 21:55:49 +0300 Subject: [PATCH 20/26] Review: a target left hovering forever, a list that could not be grabbed, and a URI that was not a file **A session that ended without an exit left the target hovered, permanently.** Three ports reach neither NativeDragAndDrop.drop() nor dragExit() on some path: an Android drop the target refused returns before the former and ACTION_DRAG_ENDED cleaned up only for the application's own drags; an AWT drop rejected on the planned action returns the same way and gets no exportDone; and iOS never implemented sessionDidEnd, which UIKit sends whether or not sessionDidExit ran -- a session cancelled inside the surface only ends. The target then stayed the framework's current one, and the next drag entering that same component was routed as a move *over* it: no enter callback ever arrived, and the component kept the ended session's answer, which for a refusal is ACTION_NONE and is deliberately never recomputed. Stuck rejected, for the life of the process. Each port clears on its own end-of-session path now, and dragEnter() no longer takes the previous target's word for it: the platform said this is an entry, so whatever is still hovered belongs to a session that is over and is exited first. That is the one place a platform forgetting to tell us cannot reach past. iOS holds the end back while a drop's asynchronous loads are still running, since that end arrives as soon as performDrop: returns and clearing there would leave the commit to find its own target gone. **A moving list could not be grabbed.** A press that lands on a momentum-scrolling container is stopping the glide, and the press it was is only dispatched by the dragStopFlag recovery in pointerDragged -- which ran *after* the native drag check, so the first motion packet handed the row to the operating system and an attempt to stop a scrolling list started an outbound drag instead. The check runs after the recovery now, which restages the press at the current position and leaves that packet below the threshold; a gesture that really does go on to drag still starts one on the next. Only reachable when resumeDragAfterScrolling is overridden to return false, which is exactly what its documentation offers, and the test overrides it. **A URI minted to carry bytes was reported as a file.** Every URI item became a file reference, which was right for the case that fix was for -- a dragged PNG *file* whose URI resolves to image/png -- and wrong for a payload of nothing but bytes: application/pdf travels as a content URI without text/uri-list ever being advertised, so calling it a file invented a representation the source never published and let a nested file-only target take a drop the PDF-capable one had been chosen for while it hovered. The two are told apart by the exporter's own record rather than guessed at, since writeAsProviderUri names what it mints and a real file keeps its own name. Both core fixes have tests, probed by disabling both at once: each fails on its own account, and the hover one reports [over] where [exit, enter] belongs. Co-Authored-By: Claude Opus 5 (1M context) --- CodenameOne/src/com/codename1/ui/Form.java | 18 +++- .../com/codename1/ui/NativeDragAndDrop.java | 15 +++ .../impl/android/AndroidImplementation.java | 36 ++++++-- .../android/AndroidNativeDragAndDrop.java | 8 ++ .../impl/javase/JavaSENativeDragAndDrop.java | 12 +++ Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 29 ++++++ .../codename1/ui/NativeDragAndDropTest.java | 92 +++++++++++++++++++ 7 files changed, 199 insertions(+), 11 deletions(-) diff --git a/CodenameOne/src/com/codename1/ui/Form.java b/CodenameOne/src/com/codename1/ui/Form.java index db939ab24fa..2c76f2121b3 100644 --- a/CodenameOne/src/com/codename1/ui/Form.java +++ b/CodenameOne/src/com/codename1/ui/Form.java @@ -4262,17 +4262,25 @@ public void pointerDragged(int x, int y) { stylusCmp.fireStylusEvent(ActionEvent.Type.PointerDrag, x, y); } } + // disable the drag stop flag if we are dragging again + boolean isScrollWheeling = Display.impl.isScrollWheeling(); + if (dragStopFlag) { + pointerPressed(x, y); + } // A press that landed on a native drag source becomes an operating system drag here, // as soon as it has moved far enough to be a drag rather than a click. From that point // the platform owns the gesture, so nothing below runs for it. + // + // After the dragStopFlag recovery above, deliberately. A press that lands on a + // momentum-scrolling container is stopping the glide, and the press it was is only + // dispatched by that recovery -- so asking first handed the row to the operating system + // on the very first motion packet, and grabbing a moving list started an outbound drag + // instead of stopping it. Run afterwards, the recovery restages the press at this + // position, which leaves this packet below the drag threshold; a gesture that really + // does go on to drag still starts one on the next. if (NativeDragAndDrop.pointerDragged(x, y)) { return; } - // disable the drag stop flag if we are dragging again - boolean isScrollWheeling = Display.impl.isScrollWheeling(); - if (dragStopFlag) { - pointerPressed(x, y); - } autoRelease(x, y); boolean localPointerPressedAgainDuringDrag = pointerPressedAgainDuringDrag; pointerPressedAgainDuringDrag = false; diff --git a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java index 4016bfa791a..58714d2065d 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java @@ -486,6 +486,21 @@ private static int dragThreshold() { /// the action a drop would perform right now, or `NativeDragOperation#ACTION_NONE` when /// nothing under the pointer will take it public static int dragEnter(int windowId, int x, int y, ClipboardContent content, int allowedActions) { + boolean stillHovered; + synchronized (LOCK) { + stillHovered = currentTarget != null; + } + if (stillHovered) { + // The platform says this is an entry, so it is one, and nothing can still be hovered + // from before it. A session that ended without an exit -- a drop the target refused, + // a drag cancelled while inside the surface -- used to leave the previous target in + // place, and the next entry was then routed as a move over it: no enter callback + // ever arrived and the component stayed at the ended session's answer, which for a + // refusal is ACTION_NONE and is deliberately never recomputed. The ports clear this + // on their own end-of-session paths as well; this is the one place that cannot be + // reached by a platform forgetting to tell us. + dragExit(windowId); + } return dragOver(windowId, x, y, content, allowedActions); } diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java index 7d78e7c35d0..f2e488d54c2 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java @@ -10668,12 +10668,23 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { } else { unnamedUris.add(uri); } - // Every URI item is a file reference as well as whatever its type made of - // it. The image branch returned before reaching this, so dragging a PNG - // *file* produced image bytes and no file at all -- and a target filtering - // on MIME_FILE accepted the hover, because the description still advertised - // text/uri-list, and was then refused the drop. - fileUris.add(uri.toString()); + // A URI item is a file reference as well as whatever its type made of it -- + // unless it is one this exporter minted to carry bytes. The image branch + // used to return before reaching this at all, so dragging a PNG *file* + // produced image bytes and no file, and a target filtering on MIME_FILE + // accepted the hover -- the description still advertised text/uri-list -- + // and was refused the drop. Adding every URI unconditionally is the other + // error: a payload of nothing but application/pdf bytes travels as a + // content URI without text/uri-list ever being advertised, and calling that + // a file both invents a representation the source never published and lets + // a nested file-only target take a drop the PDF-capable one was chosen for + // while it hovered. + // + // The two are told apart exactly, not guessed at: writeAsProviderUri names + // what it mints, and a real file keeps its own name. + if (!isGeneratedClipFile(uri)) { + fileUris.add(uri.toString()); + } continue; } } catch (Throwable t) { @@ -10723,6 +10734,19 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { return content; } + /// True when this content URI is one `#writeAsProviderUri(byte[], java.lang.String, + /// java.lang.String)` minted to carry a representation's bytes, rather than a file the + /// source published. + /// + /// Every generated file is named with the same prefix and a real one keeps its own name, so + /// this is the exporter's own record rather than an inference from the type -- which cannot + /// answer it, since a PDF published as bytes and a PDF published as a file both arrive as + /// application/pdf. + private boolean isGeneratedClipFile(Uri uri) { + String name = displayNameFor(uri); + return name != null && name.startsWith(CLIP_FILE_PREFIX); + } + /// A MIME type without its parameters, lower case, or null when there is none. private static String bareMimeType(String type) { if (type == null) { diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java b/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java index 001cb43e226..821fa47bba3 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidNativeDragAndDrop.java @@ -226,6 +226,14 @@ private static boolean handle(AndroidImplementation impl, DragEvent event) { case DragEvent.ACTION_DROP: return drop(impl, event); case DragEvent.ACTION_DRAG_ENDED: + // Whatever happened, nothing is hovered any more. Android delivers this to + // every subscribed view, and it is the only event that arrives on the paths + // that otherwise clear nothing: a drop the target refused returns before + // NativeDragAndDrop.drop(), and a drag from another application never + // reaches the dragCompleted() below. A stale target left the component + // stuck at its refusal for the next drag, which was then routed as a move + // over it rather than an entry. + NativeDragAndDrop.dragExit(0); if (exporting() != null) { // Settled *before* the operation is forgotten. Reading the allowed // actions afterwards is how this reported every move as a copy: with diff --git a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java index 1d7ff99b843..4adb3843297 100644 --- a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java +++ b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java @@ -659,6 +659,18 @@ public void drop(DropTargetDropEvent e) { } catch (Throwable ignored) { // The drop is already over; nothing left to report to. } + } finally { + // The drop is the end of the session, and AWT sends no dragExit after it: a + // rejected drop returns above without reaching NativeDragAndDrop.drop(), and a + // drag that arrived from another application has no exportDone() either, so + // without this the target stayed hovered at its own refusal and the next drag + // over it was routed as a move rather than an entry. Free after a drop that did + // go through, which clears the target itself. + try { + NativeDragAndDrop.dragExit(canvas.windowId); + } catch (Throwable ignored) { + // Nothing left to clean up. + } } } diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index 96182832591..8a1d6c3611f 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -112,6 +112,15 @@ void CN1CancelNativeDrag(void) { /// True while this application is the source of the session in progress. static BOOL cn1DraggingOut = NO; +/// A drop whose representations are still loading. +/// +/// UIKit ends the session as soon as performDrop: returns, which is long before the +/// asynchronous loads that drop depends on have answered. The end must not clear the hover +/// state while the drop that is about to use it is still in flight -- doing so made the commit +/// find its own target gone and fall back to the declarative answer, discarding whatever the +/// target's callbacks had decided. +static BOOL cn1DropInFlight = NO; + /// A drop of this application's own session onto its own surface, still loading. /// /// UIKit asks the source what happened -- dragInteraction:session:didEndWithOperation: -- as @@ -737,12 +746,29 @@ - (void)dropInteraction:(UIDropInteraction *)interaction sessionDidExit:(id)session { + // UIKit sends this whether or not sessionDidExit ran: a session cancelled, or ended, while + // still inside this surface never exits. Without it the framework kept the last target + // hovered, and the next session entering that same component was routed as a move over it + // -- inheriting the ended session's answer, with no enter callback ever arriving. + // + // Not while a drop is still loading, though. This arrives as soon as performDrop: returns, + // which is before the asynchronous loads have answered, and clearing the target there would + // leave the commit to find it gone. That path clears the hover state itself. + if (cn1DropInFlight) { + return; + } + cn1LastDropAction = CN1_DND_ACTION_NONE; + CN1NativeDragDeliverExit(); +} + - (void)dropInteraction:(UIDropInteraction *)interaction performDrop:(id)session { CGPoint point = [session locationInView:interaction.view]; const int x = (int)(point.x * scaleValue); const int y = (int)(point.y * scaleValue); const int action = cn1LastDropAction == CN1_DND_ACTION_NONE ? cn1DefaultAction(cn1AllowedActionsFor(session)) : cn1LastDropAction; + cn1DropInFlight = YES; if (session.localDragSession != nil) { cn1LocalDropInFlight = YES; cn1EndDeferred = NO; @@ -876,6 +902,9 @@ - (void)dropInteraction:(UIDropInteraction *)interaction performDrop:(id Date: Wed, 2 Sep 2026 22:28:39 +0300 Subject: [PATCH 21/26] Fix a regression, and review: a component left highlighted, and a fallback dragged as a second object **A file called cn1-clip-anything was thrown away as one of ours.** ClipboardRoundTripTest went red on the Android instrumentation suite with "file reference missing after round trip", and it was exactly right. The last change told a URI minted to carry bytes apart from a file the application published by looking at the file's name, and the file that test copies is called cn1-clip-roundtrip.txt -- so the paste threw away the very file reference it had just put on the clipboard. The name was never able to answer that question: an application may publish a file called anything at all, and the type cannot answer it either, since a PDF published as bytes and a PDF published as a file both arrive as application/pdf. Only the exporter knows which URIs it minted, so the exporter records them now. A published file goes out through FileProvider.getUriForFile, is never minted, and stays a file. The record is bounded at the last sixty-four, because a clip that has been replaced on the clipboard can no longer be pasted; one that outlives the process reads as a file, which is what it read as before any of this existed. Caught by CI rather than here: the instrumentation suite is not reachable from mvn -pl android verify, which is all "the Android port builds clean" means locally. **A drop that landed somewhere else never told the component it left.** drop() cleared the current target without dispatching NativeDragExit when the release resolved to a different component, or to none -- a quick move and let go. The old hover highlight then stayed on for good: the drop goes to somebody else, and the port's own end-of-session cleanup finds the target already cleared and has nothing left to deliver the exit to. The exit is queued before the drop, so a component losing the drag hears about it before the one taking it hears about the drop. **An Android text fallback was a second thing being dragged.** A clip item is a dragged *object*, not an alternative reading of one, so a text item beside a file item is two objects: a receiver that imports the clip gets the document *and* a stray piece of text instead of choosing the best form of one thing -- the same mistake the iOS side made and fixed by attaching the declared representations to the file's own item. The text rides on the first item carrying a URI now, and only becomes an item of its own when there is none. The reader had to change with it, or that would have silently lost the fallback on this framework's own round trip: an item carrying a URI returned before its text was ever read. It falls through now, and reads that text literally rather than through coerceToText, which for a URI goes and reads the document behind it -- a different value altogether. The exit has a test, and it reports [enter] where [enter, exit] belongs without the fix. Co-Authored-By: Claude Opus 5 (1M context) --- .../com/codename1/ui/NativeDragAndDrop.java | 12 +++ .../impl/android/AndroidImplementation.java | 99 +++++++++++++++---- .../codename1/ui/NativeDragAndDropTest.java | 31 ++++++ 3 files changed, 123 insertions(+), 19 deletions(-) diff --git a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java index 58714d2065d..f3d80871f01 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java @@ -632,7 +632,9 @@ public static void dragExit(int windowId) { public static int drop(int windowId, int x, int y, ClipboardContent content, int action) { Component target = findTarget(windowId, x, y, content, action); int accepted; + Component previous; synchronized (LOCK) { + previous = currentTarget; if (target != null && target == currentTarget) { // NOPMD CompareObjectsWithEquals // The target's own latest word, not a recomputation from the action the port // supplied. That action is by construction one event behind -- it is what the @@ -674,9 +676,19 @@ public static int drop(int windowId, int x, int y, ClipboardContent content, int overDispatchPending = false; currentAction = accepted; } + if (previous != null && previous != target) { // NOPMD CompareObjectsWithEquals + // A release that lands somewhere else -- a quick move and let go -- ends the drag + // for the component it was over, and that component has to be told. Clearing the + // target without it left the old hover highlight on for good: the drop goes to + // somebody else, and the port's own end-of-session cleanup then finds the target + // already cleared and has nothing left to deliver the exit to. + dispatch(previous, ActionEvent.Type.NativeDragExit, content, x, y, action); + } if (accepted == NativeDragOperation.ACTION_NONE) { return NativeDragOperation.ACTION_NONE; } + // Queued after the exit above, so a component losing the drag hears about it before the + // one taking it hears about the drop. dispatch(target, ActionEvent.Type.NativeDrop, content, x, y, accepted); return accepted; } diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java index f2e488d54c2..c7dae0689f1 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java @@ -10210,16 +10210,20 @@ ClipData clipDataFor(ClipboardContent content) { } } } - if (sdk >= 16 && html != null) { + // The types are recorded here, but the text does not become an item of its own yet. A + // clip item is a dragged *object*, so a text item beside a file item is two things + // being dragged at once, and a receiver that imports everything takes the document + // *and* a stray piece of text instead of choosing the best form of one thing. Where + // the clip carries a URI, the text rides on it -- see attachCarriedText below. + boolean carriesHtml = sdk >= 16 && html != null; + if (carriesHtml) { mimeTypes.add(ClipboardContent.MIME_TEXT); mimeTypes.add(ClipboardContent.MIME_HTML); - items.add(new ClipData.Item(plain, html)); } else if (plain != null) { mimeTypes.add(ClipboardContent.MIME_TEXT); if (primaryTextMime != null && !mimeTypes.contains(primaryTextMime)) { mimeTypes.add(primaryTextMime); } - items.add(new ClipData.Item(plain)); } try { addBinaryContent(content, mimeTypes, items); @@ -10227,6 +10231,9 @@ ClipData clipDataFor(ClipboardContent content) { } catch (Throwable t) { com.codename1.io.Log.e(t); } + if (carriesHtml || plain != null) { + attachCarriedText(items, plain, carriesHtml ? html : null); + } if (items.isEmpty()) { return ClipData.newPlainText("Codename One", ""); } @@ -10337,6 +10344,28 @@ private void addBinaryContent(ClipboardContent content, List mimeTypes, } } + /// Puts the clip's text on the first item that carries a URI, or makes an item of it when + /// there is none. + /// + /// Android has no notion of "an alternative reading of this object": every item is another + /// thing being dragged. A file and its text fallback therefore have to be one item, or a + /// receiver importing the clip gets two objects where the source published one. The same + /// mistake on the iOS side made a receiver import a document and a stray piece of text. + private static void attachCarriedText(List items, String plain, String html) { + for (int iter = 0; iter < items.size(); iter++) { + Uri uri = items.get(iter).getUri(); + if (uri != null) { + items.set(iter, html != null + ? new ClipData.Item(plain, html, null, uri) + : new ClipData.Item(plain, null, uri)); + return; + } + } + // Nothing to ride on, so the text is the object. First, as it was before there was + // anything else in the clip at all. + items.add(0, html != null ? new ClipData.Item(plain, html) : new ClipData.Item(plain)); + } + /// Adds the representations neither the text nor the binary pass above has taken. /// /// Byte-backed types -- a PDF, an archive, an application's own format -- become typed @@ -10414,6 +10443,7 @@ private Uri writeAsProviderUri(byte[] bytes, String extension, String mime) thro getContext().getPackageName() + ".provider", file); // Grant broadly so any paste or drop target can read the content:// URI getContext().grantUriPermission("android", uri, Intent.FLAG_GRANT_READ_URI_PERMISSION); + rememberGeneratedClipUri(uri); return uri; } @@ -10680,12 +10710,16 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { // a nested file-only target take a drop the PDF-capable one was chosen for // while it hovered. // - // The two are told apart exactly, not guessed at: writeAsProviderUri names - // what it mints, and a real file keeps its own name. + // The two are told apart by the exporter's own record of what it minted, + // not by anything about the URI or its name -- an application may publish a + // file called anything at all. if (!isGeneratedClipFile(uri)) { fileUris.add(uri.toString()); } - continue; + // No continue: an item carrying a URI carries the clip's text too, because + // that is where this exporter puts it -- a text item of its own would be a + // second object being dragged. Returning here dropped the fallback the + // source published on its own round trip. } } catch (Throwable t) { com.codename1.io.Log.e(t); @@ -10699,11 +10733,14 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { html = item.getHtmlText(); } if (plain == null) { - // Not the same test. coerceToText *derives* text from whatever the item holds, - // so an empty answer means it had nothing to give rather than that the source - // published nothing -- and accepting it would stop the search before an item - // that does carry the text. - CharSequence text = item.coerceToText(getContext()); + // Literally for an item that also carries a URI, and coerced otherwise. + // coerceToText *derives* text from whatever the item holds, which for a URI + // means going and reading the document behind it -- a different value + // altogether, and one this branch has no business producing. Where it does + // coerce, an empty answer means the item had nothing to give rather than that + // the source published nothing, so it does not stop the search. + CharSequence text = item.getUri() != null + ? item.getText() : item.coerceToText(getContext()); if (text != null && text.length() > 0) { plain = text.toString(); } @@ -10734,17 +10771,41 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { return content; } + /// The content URIs this exporter minted to carry bytes, oldest first. + /// + /// Remembered, not recognized. The file name cannot answer the question: an application may + /// publish a file of its own by any name it likes, and one called cn1-clip-roundtrip.txt is + /// exactly what the clipboard round trip publishes -- which a prefix test then threw away + /// as one of ours, losing the file reference it had just copied. The type cannot answer it + /// either, since a PDF published as bytes and a PDF published as a file both arrive as + /// application/pdf. Only the exporter knows, so the exporter records it. + /// + /// Bounded: a clip that has been replaced on the clipboard can no longer be pasted, so the + /// oldest entries are of no further use. A clip that outlives the process falls back to + /// being read as a file, which is what it was read as before any of this existed. + private static final int GENERATED_CLIP_URI_MEMORY = 64; + private static final java.util.LinkedHashSet GENERATED_CLIP_URIS = + new java.util.LinkedHashSet(); + + private static void rememberGeneratedClipUri(Uri uri) { + synchronized (GENERATED_CLIP_URIS) { + GENERATED_CLIP_URIS.remove(uri.toString()); + GENERATED_CLIP_URIS.add(uri.toString()); + java.util.Iterator oldest = GENERATED_CLIP_URIS.iterator(); + while (GENERATED_CLIP_URIS.size() > GENERATED_CLIP_URI_MEMORY && oldest.hasNext()) { + oldest.next(); + oldest.remove(); + } + } + } + /// True when this content URI is one `#writeAsProviderUri(byte[], java.lang.String, /// java.lang.String)` minted to carry a representation's bytes, rather than a file the /// source published. - /// - /// Every generated file is named with the same prefix and a real one keeps its own name, so - /// this is the exporter's own record rather than an inference from the type -- which cannot - /// answer it, since a PDF published as bytes and a PDF published as a file both arrive as - /// application/pdf. - private boolean isGeneratedClipFile(Uri uri) { - String name = displayNameFor(uri); - return name != null && name.startsWith(CLIP_FILE_PREFIX); + private static boolean isGeneratedClipFile(Uri uri) { + synchronized (GENERATED_CLIP_URIS) { + return GENERATED_CLIP_URIS.contains(uri.toString()); + } } /// A MIME type without its parameters, lower case, or null when there is none. diff --git a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java index 63a17215563..b8195782921 100644 --- a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java +++ b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java @@ -715,6 +715,37 @@ public void run() { flushSerialCalls(); } + @FormTest + void aDropThatLandsElsewhereTellsTheComponentItLeft() { + Form form = Display.getInstance().getCurrent(); + DropRecorder left = new DropRecorder(); + DropRecorder landed = new DropRecorder(); + left.setNativeDropTarget(true); + landed.setNativeDropTarget(true); + form.setLayout(new BorderLayout()); + form.add(BorderLayout.NORTH, left); + form.add(BorderLayout.SOUTH, landed); + left.setPreferredSize(new com.codename1.ui.geom.Dimension(40, 40)); + landed.setPreferredSize(new com.codename1.ui.geom.Dimension(40, 40)); + form.revalidate(); + + NativeDragAndDrop.dragEnter(0, left.getAbsoluteX() + 5, left.getAbsoluteY() + 5, + textContent("hi"), NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + assertEquals("[enter]", left.events.toString()); + + // Moved and released in one go, so the release resolves somewhere the drag never + // hovered. + NativeDragAndDrop.drop(0, landed.getAbsoluteX() + 5, landed.getAbsoluteY() + 5, + textContent("hi"), NativeDragOperation.ACTION_COPY); + flushSerialCalls(); + + assertEquals("[enter, exit]", left.events.toString(), + "the drag ended for the component it was over, and nothing else can tell it: " + + "the port's own cleanup finds the target already cleared"); + assertEquals("[drop]", landed.events.toString()); + } + @FormTest void anEntryAfterASessionThatNeverExitedIsStillAnEntry() { Form form = Display.getInstance().getCurrent(); From 46467533c05ec6693dcd3198aecc24d3a4f914d4 Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 22:39:39 +0300 Subject: [PATCH 22/26] Review: text invented for a clip that never mentioned it, and the document before its thumbnail **A binary-only drop acquired a text/plain of its own.** Every clip reported text, so a drag carrying nothing but application/pdf materialized with text/plain set to the empty string. findTarget runs again against the materialized content, so that invented type let a nested text-only component take a drop the type-capable ancestor had been chosen for while it hovered -- and that component never saw an enter event at all. It is only synthesized now when the description advertised text and no item produced it, which is the case it was there for: keeping a promise the hover made, rather than making one. **The document now goes in the clip before its thumbnail.** Review asked for binary alternatives to share one logical item, and Android cannot: ClipData.Item holds exactly one Uri, so two representations that are both bytes have no way to be one item. The platform expresses "another reading of the same object" only for text and markup, which is what the fallback already rides on. Dropping the extra representations instead is not the answer either -- they are what the description advertises, and refusing to produce them at the drop refuses the very target that accepted the hover on one of them. What order does fix is which object a receiver reading only the first item takes: the document, not a thumbnail of it. It also puts the carried text on the document rather than on the thumbnail, which is what the last change wanted and got only because nothing else was in the clip. Co-Authored-By: Claude Opus 5 (1M context) --- .../impl/android/AndroidImplementation.java | 77 ++++++++++++------- 1 file changed, 48 insertions(+), 29 deletions(-) diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java index c7dae0689f1..6a77ef67736 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java @@ -10285,32 +10285,16 @@ private void addBinaryContent(ClipboardContent content, List mimeTypes, List items) throws IOException { String authority = getContext().getPackageName() + ".provider"; - // Image bytes: prefer PNG, then JPEG, then GIF - String imageMime = null; - byte[] imageBytes = null; - String imageExt = null; - if (content.getBytes(ClipboardContent.MIME_PNG) != null) { - imageMime = ClipboardContent.MIME_PNG; - imageBytes = content.getBytes(ClipboardContent.MIME_PNG); - imageExt = "png"; - } else if (content.getBytes(ClipboardContent.MIME_JPEG) != null) { - imageMime = ClipboardContent.MIME_JPEG; - imageBytes = content.getBytes(ClipboardContent.MIME_JPEG); - imageExt = "jpg"; - } else if (content.getBytes(ClipboardContent.MIME_GIF) != null) { - imageMime = ClipboardContent.MIME_GIF; - imageBytes = content.getBytes(ClipboardContent.MIME_GIF); - imageExt = "gif"; - } - if (imageBytes != null) { - Uri imageUri = writeAsProviderUri(imageBytes, imageExt, imageMime); - if (imageUri != null) { - if (!mimeTypes.contains(imageMime)) { - mimeTypes.add(imageMime); - } - items.add(new ClipData.Item(imageUri)); - } - } + // The files first, then the byte-backed representations. Android's ClipData.Item holds + // exactly one Uri, so two representations that are both bytes cannot be one item -- the + // platform has no way to say "another reading of the same object" for them, only for + // the text and markup that attachCarriedText rides on the item below. Publishing them + // is still right: they are what the description advertises, and dropping them would + // refuse the very target that accepted the hover on one. What order fixes is which + // object a receiver reading only the first item takes -- the document, not its + // thumbnail. + // + // It is also what puts the carried text on the document rather than on the thumbnail. // File references: MIME_FILE may be a single String or a String[] Object fileData = content.getData(ClipboardContent.MIME_FILE); @@ -10342,6 +10326,33 @@ private void addBinaryContent(ClipboardContent content, List mimeTypes, items.add(new ClipData.Item(u)); } } + + // Image bytes: prefer PNG, then JPEG, then GIF + String imageMime = null; + byte[] imageBytes = null; + String imageExt = null; + if (content.getBytes(ClipboardContent.MIME_PNG) != null) { + imageMime = ClipboardContent.MIME_PNG; + imageBytes = content.getBytes(ClipboardContent.MIME_PNG); + imageExt = "png"; + } else if (content.getBytes(ClipboardContent.MIME_JPEG) != null) { + imageMime = ClipboardContent.MIME_JPEG; + imageBytes = content.getBytes(ClipboardContent.MIME_JPEG); + imageExt = "jpg"; + } else if (content.getBytes(ClipboardContent.MIME_GIF) != null) { + imageMime = ClipboardContent.MIME_GIF; + imageBytes = content.getBytes(ClipboardContent.MIME_GIF); + imageExt = "gif"; + } + if (imageBytes != null) { + Uri imageUri = writeAsProviderUri(imageBytes, imageExt, imageMime); + if (imageUri != null) { + if (!mimeTypes.contains(imageMime)) { + mimeTypes.add(imageMime); + } + items.add(new ClipData.Item(imageUri)); + } + } } /// Puts the clip's text on the first item that carries a URI, or makes an item of it when @@ -10758,9 +10769,17 @@ ClipboardContent contentFromClip(ClipData clip, ClipDescription description) { } if (plain != null) { content.setData(ClipboardContent.MIME_TEXT, plain); - } else if (!content.hasMimeType(ClipboardContent.MIME_TEXT)) { - // Every clip reports text, so a target asking for it gets "" rather than nothing -- - // but only when nothing else has supplied it. A URI the resolver typed text/plain, + } else if (!content.hasMimeType(ClipboardContent.MIME_TEXT) + && description != null && description.hasMimeType(ClipboardContent.MIME_TEXT)) { + // The clip promised text and no item produced it, so the empty string keeps that + // promise: a target that accepted the hover on text/plain would otherwise be + // refused the drop it was told it could have. Only then, though -- a clip that + // never mentioned text does not acquire it here. findTarget runs again against the + // materialized content, so inventing text/plain let a nested text-only component + // take a drop the type-capable ancestor had been chosen for while it hovered, and + // that component never saw an enter event at all. + // + // Nor over a representation that answered: a URI the resolver typed text/plain, // which is what a dragged .txt is, has already registered the document's own // contents, and writing over that handed the target an empty document. content.setData(ClipboardContent.MIME_TEXT, ""); From e40e226a71345860732d08e483a0089e90470d86 Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Wed, 2 Sep 2026 23:09:26 +0300 Subject: [PATCH 23/26] Fix a paste that counted types, and review: a preview never rendered, an in-flight flag shared by two drops **An image-only paste returned nothing.** The last change stopped every clip from acquiring a text/plain of its own, which was right, and it exposed what that padding had been holding up: getPasteDataFromClipboard decided between the legacy plain-text answer and a rich ClipboardContent by *counting* the MIME types, and one type meant plain text. With the padding gone an image-only clip counted as one, fell through to the text answer, and a paste holding a perfectly good PNG returned null. It asks what the clip holds now rather than how many names it has. ClipboardRoundTripTest caught it on all three Android legs. **A drag started from code never got the source's preview.** startDrag(source, op) documents the source as providing the default preview and then handed the operation to the port without rendering one, so Android fell back to snapshotting the entire Codename One surface and JavaSE dragged with no image at all. It renders the same snapshot the gesture path does, centred, because a drag begun in code has no grab point to offset from. **One in-flight flag spoke for two iOS drops.** A slow NSItemProvider can leave a drop assembly running while the user completes a second one, and the process-wide boolean answered for both: the first assembly's completion cleared it, so the second session's end dispatched an exit while its own loads were still running -- and its commit then found no target of its own to honour, which is the very failure the flag was added to prevent. It is a set of the sessions still loading now, and each end asks only about its own. An assembly overtaken by a newer session still commits, and that is deliberate: the framework keeps one hover state because one drag is what a platform runs, so a late commit does disturb a drag begun since -- the newer target is sent an exit and its next update re-enters it, a flicker that repairs itself. Withholding the commit would lose a drop the user actually performed, and unperformed work is worse than a repaired frame. Said in the code, where the next reader will be. The preview has a test, and its cleanup moved into a finally: a session left active wedges every test after it, so the first version of this reported one broken assertion as twenty-one failures. Co-Authored-By: Claude Opus 5 (1M context) --- .../com/codename1/ui/NativeDragAndDrop.java | 16 ++++++ .../impl/android/AndroidImplementation.java | 10 +++- Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 57 ++++++++++++++----- .../codename1/ui/NativeDragAndDropTest.java | 32 +++++++++++ 4 files changed, 99 insertions(+), 16 deletions(-) diff --git a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java index f3d80871f01..239141c7e59 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java @@ -185,6 +185,22 @@ public static boolean startDrag(Component source, NativeDragOperation op) { } op.setSource(source); op.resetPerformedAction(); + if (needsGeneratedImage(op) && source != null) { + try { + // The same snapshot the gesture path renders, because this method documents the + // source as providing the default preview and without it the ports fall back to + // something worse: Android snapshots the whole Codename One surface, and JavaSE + // drags with no image at all. + // + // Centred, because a drag begun in code has no grab point to offset from. The + // gesture path uses where the finger actually went down; there is no such place + // here, and the centre is what a preview with no anchor should hang from. + op.setGeneratedDragImage(source.getDragImage(), + source.getWidth() / 2, source.getHeight() / 2); + } catch (Throwable err) { + Log.e(err); + } + } boolean started = false; try { started = Display.impl.startNativeDrag(op); diff --git a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java index 6a77ef67736..c86f40d39a1 100644 --- a/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java +++ b/Ports/Android/src/com/codename1/impl/android/AndroidImplementation.java @@ -10604,7 +10604,15 @@ public void run() { } ClipboardContent content = contentFromClip(clip); String plain = content.getText(ClipboardContent.MIME_TEXT); - if (content.getMimeTypes().length > 1) { + // What the clip actually holds, not how many types it happens to name. + // Counting worked only because every clip used to acquire a text/plain of + // its own, empty or not: with that padding gone an image-only clip counted + // as one type, fell through to the plain-text answer, and a paste that had + // a perfectly good PNG in it returned null. + String[] types = content.getMimeTypes(); + boolean textOnly = types.length == 0 + || (types.length == 1 && ClipboardContent.MIME_TEXT.equals(types[0])); + if (!textOnly) { response[0] = content; } else { response[0] = plain != null && plain.length() > 0 ? plain : null; diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index 8a1d6c3611f..26436f5214a 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -112,14 +112,28 @@ void CN1CancelNativeDrag(void) { /// True while this application is the source of the session in progress. static BOOL cn1DraggingOut = NO; -/// A drop whose representations are still loading. +/// The drop sessions whose representations are still loading. /// -/// UIKit ends the session as soon as performDrop: returns, which is long before the -/// asynchronous loads that drop depends on have answered. The end must not clear the hover -/// state while the drop that is about to use it is still in flight -- doing so made the commit -/// find its own target gone and fall back to the declarative answer, discarding whatever the -/// target's callbacks had decided. -static BOOL cn1DropInFlight = NO; +/// UIKit ends a session as soon as performDrop: returns, which is long before the asynchronous +/// loads that drop depends on have answered. The end must not clear the hover state while the +/// drop about to use it is still in flight -- doing so made the commit find its own target gone +/// and fall back to the declarative answer, discarding whatever the target's callbacks had +/// decided. +/// +/// Per session, not a single flag. A slow NSItemProvider can leave one assembly running while +/// the user completes a second drop, and one boolean answered for both: the first assembly's +/// completion cleared it, so the second session's end went on to dispatch an exit while its own +/// loads were still running -- and its commit then found no target of its own to honour, which +/// is the very failure the flag exists to prevent. +/// +/// The table holds its sessions weakly; each assembly's own completion block is what keeps its +/// session alive for as long as the answer is needed. +static NSHashTable* cn1LoadingDropSessions = nil; + +static BOOL cn1DropIsLoading(id session) { + return session != nil && cn1LoadingDropSessions != nil + && [cn1LoadingDropSessions containsObject:session]; +} /// A drop of this application's own session onto its own surface, still loading. /// @@ -752,10 +766,11 @@ - (void)dropInteraction:(UIDropInteraction *)interaction sessionDidEnd:(id Date: Wed, 2 Sep 2026 23:49:32 +0300 Subject: [PATCH 24/26] The gesture never started a native drag, and the tests were driving a path no port takes **A one-finger drag reaches a Form through pointerDragged(int[], int[]).** CodenameOneImplementation wraps a single pointer into one-element arrays and Display dispatches those, and Form and Window each implement that overload separately rather than calling the scalar one. The hook was only in the scalar overload, so a gesture never began a native drag on any port that starts one itself -- which is every port except iOS, whose own recognizer announces the session through dragSessionStarted(). JavaSE and Android could not drag out at all. The tests did not catch it because they called the scalar overload directly, so thirty-three of them passed against a path no port takes. They drive the array overload now, through a helper that says why, and removing the new hook fails thirty-one of them. A second pointer is refused: two fingers are a pinch or a two-finger scroll, not a drag to hand to the operating system. The scalar overload keeps its hook and a test of its own, because it is public API an application may call and the two must not drift apart again. **An older iOS drop could complete a newer drag.** The local-drop completion state was still process-wide, so an external drop that was still loading when a local drag was dropped saw that local session's flag and completed its source with the external drop's action. Only the assembly that set the state consumes it now. Two local drops cannot overlap -- the framework runs one drag at a time and a local drop's completion is what ends it -- so that is the whole of the exposure. **The drop event reported the chosen action as the source's whole mask.** getAllowedActions() is documented as what the source permits, and the drop was constructed with the single action the target had settled on, so a listener could not tell a move-only source from one that offered both and chose a move -- and, because the event derives its default from that mask, a target handed a move read getAcceptedAction() as a copy. The drop now carries the mask the drag advertised and the action actually being performed, which are two questions. All three have tests, each probed on its own. Co-Authored-By: Claude Opus 5 (1M context) --- CodenameOne/src/com/codename1/ui/Form.java | 14 ++ .../com/codename1/ui/NativeDragAndDrop.java | 38 ++++- CodenameOne/src/com/codename1/ui/Window.java | 11 ++ Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 10 +- .../codename1/ui/NativeDragAndDropTest.java | 131 +++++++++++++++--- 5 files changed, 183 insertions(+), 21 deletions(-) diff --git a/CodenameOne/src/com/codename1/ui/Form.java b/CodenameOne/src/com/codename1/ui/Form.java index 2c76f2121b3..484e6e23f9b 100644 --- a/CodenameOne/src/com/codename1/ui/Form.java +++ b/CodenameOne/src/com/codename1/ui/Form.java @@ -4359,6 +4359,20 @@ public void pointerDragged(int[] x, int[] y) { if (dragStopFlag) { pointerPressed(x, y); } + // The same hook the scalar overload runs, and the one that matters: an ordinary + // one-finger drag reaches a Form through *this* method. CodenameOneImplementation wraps + // its coordinates into one-element arrays and Display dispatches them here, and this + // overload is a separate implementation rather than a call to the scalar one -- so with + // the hook only there, a gesture never started a native drag on any port that begins + // one itself, which is every port except the one whose operating system owns the + // gesture. Placed after the dragStopFlag recovery for the reason the scalar overload + // gives. + // + // One pointer only. A second finger makes this a pinch or a two-finger scroll, and + // handing that to the operating system as a drag is not what the user is doing. + if (x.length == 1 && NativeDragAndDrop.pointerDragged(x[0], y[0])) { + return; + } autoRelease(x[0], y[0]); boolean localPointerPressedAgainDuringDrag = pointerPressedAgainDuringDrag; if (pointerDraggedListeners != null && pointerDraggedListeners.hasListeners()) { diff --git a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java index 239141c7e59..5c12f82c0db 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java @@ -103,6 +103,13 @@ public final class NativeDragAndDrop { /// the session is not asked again on every drag event of the same gesture. private static boolean startOffered; + /// The action mask the drag last advertised, so the drop event can report what the + /// *source* permits rather than the one action the target settled on. The port hands + /// drop() a single action, not a mask, and NativeDropEvent#getAllowedActions() is + /// documented as the source's -- a listener could not otherwise tell a move-only source + /// from one that offered both and chose a move. + private static int advertisedActions; + /// The session the operating system is currently running, or null. private static NativeDragOperation active; @@ -566,6 +573,7 @@ public static int dragOver(int windowId, int x, int y, ClipboardContent content, boolean dispatchOver = false; int answer; synchronized (LOCK) { + advertisedActions = allowedActions; previous = currentTarget; changed = previous != target; // NOPMD CompareObjectsWithEquals if (changed) { @@ -618,6 +626,7 @@ public static void dragExit(int windowId) { currentTarget = null; targetGeneration++; currentAction = NativeDragOperation.ACTION_NONE; + advertisedActions = NativeDragOperation.ACTION_NONE; } dispatch(previous, ActionEvent.Type.NativeDragExit, null, 0, 0, NativeDragOperation.ACTION_NONE); } @@ -649,7 +658,13 @@ public static int drop(int windowId, int x, int y, ClipboardContent content, int Component target = findTarget(windowId, x, y, content, action); int accepted; Component previous; + int advertised; synchronized (LOCK) { + // What the drag has been advertising all along. A drop arriving with no drag + // event before it -- which no real port does -- has only the port's one action + // to report. + advertised = advertisedActions == NativeDragOperation.ACTION_NONE + ? action : advertisedActions; previous = currentTarget; if (target != null && target == currentTarget) { // NOPMD CompareObjectsWithEquals // The target's own latest word, not a recomputation from the action the port @@ -703,9 +718,12 @@ public static int drop(int windowId, int x, int y, ClipboardContent content, int if (accepted == NativeDragOperation.ACTION_NONE) { return NativeDragOperation.ACTION_NONE; } - // Queued after the exit above, so a component losing the drag hears about it before the - // one taking it hears about the drop. - dispatch(target, ActionEvent.Type.NativeDrop, content, x, y, accepted); + // Queued after the exit above, so a component losing the drag hears about it before + // the one taking it hears about the drop. The event carries the source's whole mask + // and the action actually being performed -- different questions that used to get + // the same answer, so the drop reported the chosen action as though it were all the + // source had ever allowed. + dispatch(target, ActionEvent.Type.NativeDrop, content, x, y, advertised, accepted); return accepted; } @@ -764,6 +782,7 @@ public static void dragCompleted(final int performedAction) { targetGeneration++; currentAction = NativeDragOperation.ACTION_NONE; overDispatchPending = false; + advertisedActions = NativeDragOperation.ACTION_NONE; } if (op == null) { return; @@ -854,6 +873,12 @@ private static int preferredAction(int actions) { /// give the operating system. private static void dispatch(final Component target, final ActionEvent.Type type, final ClipboardContent content, final int x, final int y, final int allowedActions) { + dispatch(target, type, content, x, y, allowedActions, NativeDragOperation.ACTION_NONE); + } + + private static void dispatch(final Component target, final ActionEvent.Type type, + final ClipboardContent content, final int x, final int y, final int allowedActions, + final int performedAction) { if (target == null) { if (type == ActionEvent.Type.NativeDragOver) { synchronized (LOCK) { @@ -873,6 +898,13 @@ private static void dispatch(final Component target, final ActionEvent.Type type public void run() { try { NativeDropEvent ev = new NativeDropEvent(target, type, content, x, y, allowedActions, local); + if (type == ActionEvent.Type.NativeDrop + && performedAction != NativeDragOperation.ACTION_NONE) { + // The action the drop is performing, which is not what the event + // would default to: a source allowing both defaults to a copy, so a + // target handed a move read getAcceptedAction() as a copy. + ev.accept(performedAction); + } if (type == ActionEvent.Type.NativeDragOver || type == ActionEvent.Type.NativeDragEnter) { // Read as this runs, not when it was queued. A drag event can arrive // before the enter callback ahead of it in the queue has run, and diff --git a/CodenameOne/src/com/codename1/ui/Window.java b/CodenameOne/src/com/codename1/ui/Window.java index 8b2e9a2c315..96ec0e83ea6 100644 --- a/CodenameOne/src/com/codename1/ui/Window.java +++ b/CodenameOne/src/com/codename1/ui/Window.java @@ -3358,6 +3358,17 @@ public void pointerDragged(int x, int y) { /// pressed child gets an ordinary one-finger drag and never its `pinch` callbacks. @Override public void pointerDragged(int[] x, int[] y) { + // The same hook the scalar overload runs, and the one an ordinary one-finger drag + // actually reaches: Display wraps a single pointer into one-element arrays and + // dispatches them here, and this overload is a separate implementation rather than a + // call to the scalar one. With the hook only there a gesture never began a native drag + // on any port that starts one itself. + // + // One pointer only: a second finger makes this a pinch or a two-finger scroll, which is + // not a drag to hand to the operating system. + if (x.length == 1 && NativeDragAndDrop.pointerDragged(x[0], y[0])) { + return; + } // The same listener block the scalar overload runs. Adding it there only // meant a gesture stopped notifying window listeners the moment it became // multi touch, which is where pull to refresh loses its updates. diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index 26436f5214a..2bb854261f7 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -790,7 +790,13 @@ - (void)dropInteraction:(UIDropInteraction *)interaction performDrop:(id Date: Thu, 3 Sep 2026 00:14:20 +0300 Subject: [PATCH 25/26] Review: a target accepting more than it declared, alternatives lost with the file, and text read in the wrong encoding **A copy-only target could accept a move.** accept() validated against the source's mask alone, so a component that declared setAcceptedDropActions(ACTION_COPY) and then asked for a move in a listener got the move -- and a move is the source deleting its copy. The declaration is what made the component eligible for the drag and what the framework's own default already respects; a listener widening past it is a contradiction, and the safe reading of a contradiction is the refusal accept() already gives for an action the source never offered. **A file that would not materialize took every alternative with it.** The loop that loads a provider's other representations sat inside the successful-URL branch of the file load, so a cloud-backed document that could not be produced left the drop with none of the text or binary fallbacks the hover had already promised, and the target that accepted the drag got nothing at all. The alternatives load whether or not the file does; where there is no document, each simply gets a copy of its own rather than sharing one. **A file-backed text alternative was read as UTF-8 whatever it said it was.** A representation handed over as a path keeps only that path, and the Java side decoded every text one as UTF-8 -- so a public.utf16-plain-text alternative arrived as rubbish, while the very same representation read as data came through intact, because that path keeps its type identifier. The charset the identifier declares now travels with the file. The bridge signature changed to carry it, so the mangled callback name changed with it: header, IOSNative.m and the Java method move together or the symbol quietly stops matching and the method is dropped as unused. The first has a test; the two iOS ones are compile-checked on all six Apple slices and by the offline signature verifier, which is what this tree can check without a device. Co-Authored-By: Claude Opus 5 (1M context) --- .../src/com/codename1/ui/NativeDropEvent.java | 18 ++++- Ports/iOSPort/nativeSources/CN1DragAndDrop.h | 6 +- Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 75 +++++++++++++------ Ports/iOSPort/nativeSources/IOSNative.m | 7 +- .../codename1/impl/ios/IOSImplementation.java | 19 ++++- .../codename1/ui/NativeDragAndDropTest.java | 24 ++++++ 6 files changed, 119 insertions(+), 30 deletions(-) diff --git a/CodenameOne/src/com/codename1/ui/NativeDropEvent.java b/CodenameOne/src/com/codename1/ui/NativeDropEvent.java index 2ab27074a1f..f20003404e1 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDropEvent.java +++ b/CodenameOne/src/com/codename1/ui/NativeDropEvent.java @@ -76,7 +76,7 @@ public final class NativeDropEvent extends ActionEvent { this.content = content; this.allowedActions = allowedActions; this.local = local; - this.acceptedAction = defaultAction(allowedActions); + this.acceptedAction = defaultAction(permittedActions()); } /// Picks the action a target that expresses no preference gets: a copy when the source @@ -143,7 +143,21 @@ public int getAcceptedAction() { /// - `action`: one of `NativeDragOperation#ACTION_COPY`, `NativeDragOperation#ACTION_MOVE` /// or `NativeDragOperation#ACTION_LINK` public void accept(int action) { - acceptedAction = (action & allowedActions) == action ? action : NativeDragOperation.ACTION_NONE; + int permitted = permittedActions(); + acceptedAction = (action & permitted) == action ? action : NativeDragOperation.ACTION_NONE; + } + + /// What may actually be agreed to here: what the source offers, narrowed by what the + /// target said it accepts. + /// + /// Both halves, because a target that declared itself copy-only and then accepted a move + /// in a listener used to have that move honoured -- and a move is the source deleting its + /// copy. The declaration is what made this component eligible for the drag in the first + /// place, and it is what the framework's own default already respects; a listener widening + /// past it is a contradiction, and the safe reading of a contradiction is a refusal, which + /// is what accept() already does for an action the source never offered. + private int permittedActions() { + return target == null ? allowedActions : allowedActions & target.getAcceptedDropActions(); } /// Refuses the drag, so the user sees a "no drop" cursor over this component and no drop diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.h b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h index f0306390c35..4bff00dc10b 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.h +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.h @@ -144,7 +144,11 @@ void CN1NativeDragDeliverDropAdd(NSString* mimeType, NSString* text, NSData* bin /// Loading that as data would read the whole document into memory on top of the copy this /// already makes -- fatal for a large one -- so the type is named against the copy instead and /// read only if a target asks for it. -void CN1NativeDragDeliverDropAddFile(NSString* mimeType, NSString* path); +/// +/// `charset` is the encoding the representation's uniform type identifier declared, or nil +/// when it declared none. Without it the Java side has only the path and reads every text +/// representation as UTF-8, which turns a public.utf16-plain-text alternative into rubbish. +void CN1NativeDragDeliverDropAddFile(NSString* mimeType, NSString* path, NSString* charset); /// Delivers the assembled drop and returns the action actually accepted, or 0 when nothing took /// it. diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index 2bb854261f7..59b8b1758d2 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -505,12 +505,29 @@ - (void)dealloc { /// /// Anything the identifier does not pin down is handed to NSString's own detection rather than /// assumed, and only a representation nothing can read at all comes back nil. -static NSString* cn1TextFromData(NSData* data, NSString* uti) { - NSStringEncoding declared = 0; +/// The charset a uniform type identifier declares, by the name java.nio understands, or nil +/// when the identifier says nothing about the encoding. +/// +/// A representation handed over as a file keeps only its path, so this is what travels with +/// it: the Java side reads the bytes later and would otherwise have to assume UTF-8, which +/// turns a public.utf16-plain-text alternative into rubbish. +static NSString* cn1CharsetNameForUti(NSString* uti) { if ([uti isEqualToString:@"public.utf16-plain-text"] || [uti isEqualToString:@"public.utf16-external-plain-text"]) { + return @"UTF-16"; + } + if ([uti isEqualToString:@"public.utf8-plain-text"]) { + return @"UTF-8"; + } + return nil; +} + +static NSString* cn1TextFromData(NSData* data, NSString* uti) { + NSStringEncoding declared = 0; + NSString* charset = cn1CharsetNameForUti(uti); + if ([charset isEqualToString:@"UTF-16"]) { declared = NSUTF16StringEncoding; - } else if ([uti isEqualToString:@"public.utf8-plain-text"]) { + } else if ([charset isEqualToString:@"UTF-8"]) { declared = NSUTF8StringEncoding; } if (declared != 0) { @@ -811,9 +828,11 @@ - (void)dropInteraction:(UIDropInteraction *)interaction performDrop:(id 0) { CN1NativeDragDeliverDropAdd(@"application/x-file-list", diff --git a/Ports/iOSPort/nativeSources/IOSNative.m b/Ports/iOSPort/nativeSources/IOSNative.m index 694d8013023..4c8b780aaa9 100644 --- a/Ports/iOSPort/nativeSources/IOSNative.m +++ b/Ports/iOSPort/nativeSources/IOSNative.m @@ -1136,11 +1136,12 @@ void CN1NativeDragDeliverDropAdd(NSString* mimeType, NSString* text, NSData* bin binary == nil ? JAVA_NULL : nsDataToByteArr(binary)); } -void CN1NativeDragDeliverDropAddFile(NSString* mimeType, NSString* path) { - com_codename1_impl_ios_IOSImplementation_nativeDropAddFileCallback___java_lang_String_java_lang_String( +void CN1NativeDragDeliverDropAddFile(NSString* mimeType, NSString* path, NSString* charset) { + com_codename1_impl_ios_IOSImplementation_nativeDropAddFileCallback___java_lang_String_java_lang_String_java_lang_String( CN1_THREAD_GET_STATE_PASS_ARG fromNSString(CN1_THREAD_GET_STATE_PASS_ARG mimeType), - fromNSString(CN1_THREAD_GET_STATE_PASS_ARG path)); + fromNSString(CN1_THREAD_GET_STATE_PASS_ARG path), + fromNSString(CN1_THREAD_GET_STATE_PASS_ARG charset)); } NSData* CN1NativeDragDeliverResolve(NSString* mimeType, int sessionId) { diff --git a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java index 7f969551189..b7fc8483c12 100644 --- a/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java +++ b/Ports/iOSPort/src/com/codename1/impl/ios/IOSImplementation.java @@ -9333,7 +9333,8 @@ public static void nativeDropAddCallback(String mimeType, String text, byte[] bi /// and a target may filter on either. Promising the type against the copy rather than /// loading it keeps the two in agreement without reading a large document into memory on /// top of copying it -- the bytes are only read if something asks for that type. - public static void nativeDropAddFileCallback(String mimeType, final String path) { + public static void nativeDropAddFileCallback(String mimeType, final String path, + final String charset) { if (pendingDrop == null || mimeType == null || mimeType.length() == 0 || path == null || path.length() == 0 || pendingDrop.hasMimeType(mimeType)) { return; @@ -9357,8 +9358,13 @@ public Object getClipboardData(String requested) { // offers a plain text representation beside its file URL, and answering // that with bytes made getText() and NativeDropEvent.getText() null for a // type the drop had just accepted. + // + // In the encoding the representation's own type identifier declared, when + // it declared one. This side never sees that identifier -- it has a path + // and a MIME type -- so assuming UTF-8 turned a UTF-16 alternative into + // rubbish while the same representation read as data came through intact. if (bytes != null && requested != null && requested.startsWith("text/")) { - return new String(bytes, "UTF-8"); + return new String(bytes, charsetOrUtf8(charset)); } return bytes; } catch (Throwable err) { @@ -9369,6 +9375,15 @@ public Object getClipboardData(String requested) { }); } + /// The named charset when this platform has it, and UTF-8 otherwise -- which is what the + /// unnamed case means anyway. + private static String charsetOrUtf8(String charset) { + if (charset == null || charset.length() == 0) { + return "UTF-8"; + } + return charset; + } + /// Invoked from CN1DragAndDrop.m once every representation has arrived. Returns the action /// accepted, or zero when nothing under the pointer took it. public static int nativeDropCommitCallback(int x, int y, int action) { diff --git a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java index 705780560d2..221f65cfd32 100644 --- a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java +++ b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java @@ -877,6 +877,30 @@ void grabbingAScrollingContainerStopsItRatherThanDraggingOut() { } } + @FormTest + void aTargetCannotAcceptMoreThanItDeclared() { + Form form = Display.getInstance().getCurrent(); + DropRecorder target = addTarget(form); + // Declared copy-only, and its listener asks for a move anyway. + target.setAcceptedDropActions(NativeDragOperation.ACTION_COPY); + target.rejectAction = NativeDragOperation.ACTION_MOVE; + int x = target.getAbsoluteX() + 5; + int y = target.getAbsoluteY() + 5; + int both = NativeDragOperation.ACTION_COPY | NativeDragOperation.ACTION_MOVE; + + NativeDragAndDrop.dragEnter(0, x, y, textContent("hi"), both); + flushSerialCalls(); + int answer = NativeDragAndDrop.dragOver(0, x, y, textContent("hi"), both); + + assertEquals(NativeDragOperation.ACTION_NONE, answer, + "a move is the source deleting its copy, and this target said it does not do " + + "moves -- honouring the listener over the declaration that made the " + + "component eligible would destroy data on its word"); + + NativeDragAndDrop.dragExit(0); + flushSerialCalls(); + } + @FormTest void theDropEventReportsWhatTheSourceAllowedAndWhatIsHappening() { Form form = Display.getInstance().getCurrent(); From 7f71f7ff0f9728c8aeff90f10d6ac7070836b65e Mon Sep 17 00:00:00 2001 From: Shai Almog <67850168+shai-almog@users.noreply.github.com> Date: Thu, 3 Sep 2026 00:40:17 +0300 Subject: [PATCH 26/26] Review: a staged drag that outlived its press, iOS files under one spelling, and what getAllowedActions owes **startDrag() left the press's staged operation behind.** A long press handler starting a drag of its own made that drag active and left what the press had staged in place, so the release that followed cancelled the port's staging underneath a session already running -- and a later press at the very same pixel looked like the old one to isStagedFor, which skipped asking the component and dragged the stale payload. The explicit start spends it. **An iOS file drag was only offered under one of its two names.** A provider vending public.file-url advertised application/x-file-list and not text/uri-list, so a component filtered to MIME_URI_LIST refused an ordinary drag out of Files while the same component took one from the Finder and from a file manager on Android. Both spellings now, in the session description and in what the drop materializes, CRLF separated as RFC 2483 has it and as the other two ports write and read it. **getAllowedActions() reports what may be accepted, and that is the right answer.** Review reads the JavaSE adapter narrowing the mask by the drop modifier as losing the source's real permissions. It is deliberate, and the contract is what needed fixing rather than the adapter: the value's stated purpose is to be what accept() will take, and a source offering copy and move while the user holds the key for a move is offering a move *now*. Reporting both would invite a target to accept the copy the user has just said they do not want, and nothing on the port side could then tell the framework which of the two to believe. It is also the same answer on the enter, the over and the drop, so a target reads one thing throughout a drag. Said on the method and again where the adapter narrows. Nothing carries "what the source would have permitted had the user not chosen", and the documentation now says so rather than implying it. The staged-operation fix has a test; without it the release cancels staging under a live session. Co-Authored-By: Claude Opus 5 (1M context) --- .../com/codename1/ui/NativeDragAndDrop.java | 13 +++++++ .../src/com/codename1/ui/NativeDropEvent.java | 15 +++++++- .../impl/javase/JavaSENativeDragAndDrop.java | 6 +++ Ports/iOSPort/nativeSources/CN1DragAndDrop.m | 24 ++++++++++-- .../codename1/ui/NativeDragAndDropTest.java | 37 +++++++++++++++++++ 5 files changed, 90 insertions(+), 5 deletions(-) diff --git a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java index 5c12f82c0db..b0251163773 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java +++ b/CodenameOne/src/com/codename1/ui/NativeDragAndDrop.java @@ -216,6 +216,19 @@ public static boolean startDrag(Component source, NativeDragOperation op) { // the gesture simply stays a lightweight one. Log.e(err); } + synchronized (LOCK) { + if (started) { + // Whatever the press staged is spent: the application has started a drag of + // its own and the gesture belongs to that. Leaving it staged had the release + // that follows cancel the port's staging underneath a session already + // running, and left a later press at the very same pixel looking like this + // one to isStagedFor -- so it skipped asking the component and dragged the + // stale payload. + pending = null; + pendingSource = null; + startOffered = false; + } + } if (!started) { synchronized (LOCK) { if (active == op) { // NOPMD CompareObjectsWithEquals diff --git a/CodenameOne/src/com/codename1/ui/NativeDropEvent.java b/CodenameOne/src/com/codename1/ui/NativeDropEvent.java index f20003404e1..9c5480cf5c9 100644 --- a/CodenameOne/src/com/codename1/ui/NativeDropEvent.java +++ b/CodenameOne/src/com/codename1/ui/NativeDropEvent.java @@ -104,7 +104,20 @@ public Component getTarget() { return target; } - /// Returns the bit set of actions the drag source permits. + /// Returns the bit set of actions that may be agreed to here and now, which is what + /// `#accept(int)` will take. + /// + /// The source's permissions are where it starts, narrowed by anything that has since + /// narrowed them. A desktop modifier is the usual one: a source offering a copy and a + /// move while the user holds the key for a move is offering a move *now*, and reporting + /// both would invite a target to accept the copy the user has just said they do not + /// want. That is the question this answers -- what may be accepted -- and it is the same + /// answer on the enter, the over and the drop, so a target reads one thing throughout a + /// drag rather than one thing while hovering and another at the end. + /// + /// It is not a way to ask what the source would have permitted in the absence of the + /// user's choice; nothing carries that, and a target that acted on it would be acting + /// against the choice. public int getAllowedActions() { return allowedActions; } diff --git a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java index 4adb3843297..e9f2eb13f3b 100644 --- a/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java +++ b/Ports/JavaSE/src/com/codename1/impl/javase/JavaSENativeDragAndDrop.java @@ -681,6 +681,12 @@ public void drop(DropTargetDropEvent e) { /// ask for a move changed nothing, because getDropAction, which is where AWT records /// that choice, was never read. The user's choice wins where the source allows it, and /// the full mask stands where it does not. + /// + /// The narrowed set is what the framework reports as + /// `com.codename1.ui.NativeDropEvent#getAllowedActions()`, and that is the answer that + /// method owes: what may be accepted here and now. Reporting the unnarrowed mask instead + /// would invite a target to accept the copy the user has just said they do not want, and + /// nothing on this side could then tell the framework which of the two to believe. private int allowedActionsFor(DropTargetDragEvent e) { int sourceActions = fromAwtActions(e.getSourceActions()); int chosen = fromAwtActions(e.getDropAction()) & sourceActions; diff --git a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m index 59b8b1758d2..30ee68a5680 100644 --- a/Ports/iOSPort/nativeSources/CN1DragAndDrop.m +++ b/Ports/iOSPort/nativeSources/CN1DragAndDrop.m @@ -358,10 +358,17 @@ static UIDropOperation cn1DropOperationFor(int action) { } } // A provider that can vend a file is a file drag whatever else it also offers, which is - // how a document dragged out of Files reaches a target that asked for files. - if ([item.itemProvider hasItemConformingToTypeIdentifier:@"public.file-url"] - && ![mimes containsObject:@"application/x-file-list"]) { - [mimes addObject:@"application/x-file-list"]; + // how a document dragged out of Files reaches a target that asked for files. Both + // spellings of that, as the other two ports publish them: a component filtered to + // MIME_URI_LIST refused an ordinary drag out of Files while the same component took + // it from the Finder and from a file manager on Android. + if ([item.itemProvider hasItemConformingToTypeIdentifier:@"public.file-url"]) { + if (![mimes containsObject:@"application/x-file-list"]) { + [mimes addObject:@"application/x-file-list"]; + } + if (![mimes containsObject:@"text/uri-list"]) { + [mimes addObject:@"text/uri-list"]; + } } } return [mimes componentsJoinedByString:@"\n"]; @@ -957,6 +964,15 @@ - (void)dropInteraction:(UIDropInteraction *)interaction performDrop:(id 0) { CN1NativeDragDeliverDropAdd(@"application/x-file-list", [files componentsJoinedByString:@"\n"], nil); + // The same files as a URI list, which is what the session advertised and so what + // a target filtered to it accepted the hover on. RFC 2483 separates them with + // CRLF, which is what the other ports write and read. + NSMutableString* uris = [NSMutableString string]; + for (NSString* path in files) { + [uris appendString:[[NSURL fileURLWithPath:path] absoluteString]]; + [uris appendString:@"\r\n"]; + } + CN1NativeDragDeliverDropAdd(@"text/uri-list", uris, nil); } // An assembly overtaken by a newer session still commits. The framework keeps one hover // state, because one drag is what a platform runs, so a late commit does disturb a drag diff --git a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java index 221f65cfd32..20d7974106a 100644 --- a/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java +++ b/maven/core-unittests/src/test/java/com/codename1/ui/NativeDragAndDropTest.java @@ -877,6 +877,43 @@ void grabbingAScrollingContainerStopsItRatherThanDraggingOut() { } } + @FormTest + void aDragStartedInCodeSpendsWhatThePressStaged() { + implementation.resetNativeDragState(); + implementation.setNativeDragAndDropSupported(true); + try { + Form form = Display.getInstance().getCurrent(); + Container source = new Container(); + source.setNativeDragOperation(new NativeDragOperation("staged by the press")); + form.setLayout(new BorderLayout()); + form.add(BorderLayout.CENTER, source); + form.revalidate(); + + int x = source.getAbsoluteX() + 10; + int y = source.getAbsoluteY() + 10; + form.pointerPressed(x, y); + assertNotNull(implementation.getPreparedNativeDrag(), "the press staged one"); + + // A long press handler starting a drag of its own, which is what this entry point + // is for. + NativeDragOperation started = new NativeDragOperation("started in code"); + assertTrue(NativeDragAndDrop.startDrag(source, started)); + + int cancelledBefore = implementation.getCancelledNativeDrags(); + form.pointerReleased(x, y); + assertEquals(cancelledBefore, implementation.getCancelledNativeDrags(), + "the release must not cancel the port's staging underneath a session that " + + "is already running"); + assertSame(started, NativeDragAndDrop.getActiveDrag(), + "and the drag started in code is still the running one"); + } finally { + NativeDragAndDrop.dragCompleted(NativeDragOperation.ACTION_NONE); + flushSerialCalls(); + implementation.setNativeDragAndDropSupported(false); + implementation.resetNativeDragState(); + } + } + @FormTest void aTargetCannotAcceptMoreThanItDeclared() { Form form = Display.getInstance().getCurrent();