Skip to content

Data converter Widget - #76

Merged
lachlangrose merged 20 commits into
dev-0.1.12from
data_converter_tool
Jan 30, 2026
Merged

Data converter Widget#76
lachlangrose merged 20 commits into
dev-0.1.12from
data_converter_tool

Conversation

@rabii-chaarani

@rabii-chaaranirabii-chaarani commented Jan 27, 2026

Copy link
Copy Markdown
Member

No description provided.

CopilotAI reviewed Jan 27, 2026
View reviewed changes

CopilotAI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request introduces a comprehensive "Data Converter Widget" that enables users to convert geological datasets for use within LoopStructural through both automatic and manual conversion workflows.

Changes:

  • Added a new data conversion widget with automatic and manual conversion modes
  • Implemented utility functions to convert GeoPandas GeoDataFrames and pandas DataFrames to QGIS vector layers
  • Integrated the data conversion widget into the main Loop widget interface
  • Added tests for the NTGS configuration model

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 17 comments.

Show a summary per file
FileDescription
tests/unit/gui/test_ntgs_configuration_model.pyUnit tests for configuration model (imports incorrect class names)
loopstructural/main/vectorLayerWrapper.pyNew functions to convert DataFrames/GeoDataFrames to QGIS layers
loopstructural/gui/loop_widget.pyIntegration of data conversion widget into main interface
loopstructural/gui/data_conversion/data_conversion_widget.pyMain widget implementation with automatic and manual conversion modes
loopstructural/gui/data_conversion/configuration.pyConfiguration state management for NTGS data types
loopstructural/gui/data_conversion/init.pyPackage initialization exposing DataConversionWidget

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadtests/unit/gui/test_ntgs_configuration_model.py
@@ -0,0 +1,33 @@
import pytest

from loopstructural.gui.data_conversion.configuration import NtgsConfig, NtgsConfigurationModel

CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test imports NtgsConfig and NtgsConfigurationModel, but the actual configuration module defines these classes as Config and ConfigurationState. This import will fail at runtime. The imports should be updated to match the actual class names in loopstructural/gui/data_conversion/configuration.py.

Copilot uses AI. Check for mistakes.
Comment threadloopstructural/main/vectorLayerWrapper.py
Comment on lines +750 to +764
for _, row in df.iterrows():
feat = QgsFeature(layer.fields())
attrs = []
for column in df.columns:
val = row[column]
if pd.isna(val):
val = None
elif isinstance(val, np.generic):
try:
val = val.item()
except Exception:
pass
attrs.append(val)
feat.setAttributes(attrs)
features.append(feat)

CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using iterrows() is generally inefficient for large DataFrames as it creates a new Series object for each row. For better performance with large datasets, consider using itertuples() or vectorized operations where possible.

Copilot uses AI. Check for mistakes.
Comment threadloopstructural/gui/loop_widget.py Outdated
self.data_conversion_widget = DataConversionWidget(
self,
data_manager=self.data_manager,
project=self.data_manager.project if self.data_manager else None,

CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accessing self.data_manager.project could raise an AttributeError if data_manager is not None but lacks a project attribute. Consider using getattr(self.data_manager, 'project', None) for safer attribute access.

Suggested change
project=self.data_manager.projectifself.data_managerelseNone,
project=getattr(self.data_manager, "project", None)ifself.data_managerelseNone,

Copilot uses AI. Check for mistakes.
Comment threadloopstructural/gui/data_conversion/configuration.py Outdated
Comment threadloopstructural/main/vectorLayerWrapper.py
return layer


def QgsLayerFromDataFrame(dataframe, layer_name: str = "Converted Table"):

CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function name QgsLayerFromDataFrame uses PascalCase, which is inconsistent with existing functions in this file like qgsLayerToGeoDataFrame, qgsLayerToDataFrame, and qgsRasterToGdalDataset that use camelCase. Consider renaming to qgsLayerFromDataFrame to maintain consistency with the existing naming convention in this file.

Suggested change
defQgsLayerFromDataFrame(dataframe, layer_name: str="Converted Table"):
defqgsLayerFromDataFrame(dataframe, layer_name: str="Converted Table"):

Copilot uses AI. Check for mistakes.
Comment threadloopstructural/main/vectorLayerWrapper.py
try:
if temp.createFromWkt(text):
return temp
except Exception:

CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Copilot uses AI. Check for mistakes.
Comment threadloopstructural/main/vectorLayerWrapper.py
Comment threadloopstructural/main/vectorLayerWrapper.py
temp = QgsCoordinateReferenceSystem(crs_info)
if temp.isValid():
return temp
except Exception:

CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change
exceptException:
exceptException:
# Ignore failures when interpreting CRS from string; fall back to default CRS.

Copilot uses AI. Check for mistakes.
Comment threadloopstructural/main/vectorLayerWrapper.py
Comment on lines +713 to +714
except Exception:
pass

CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change
exceptException:
pass
exceptExceptionasexc:
logger.debug("Failed to convert numpy scalar %r using .item(): %s", val, exc)

Copilot uses AI. Check for mistakes.
Comment threadloopstructural/main/vectorLayerWrapper.py
Comment on lines +760 to +761
except Exception:
pass

CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change
exceptException:
pass
exceptExceptionasexc:
# Fall back to the original NumPy scalar if conversion fails.
logger.debug("Failed to convert NumPy scalar %r with .item(): %s", val, exc)

Copilot uses AI. Check for mistakes.
@lachlangrose

lachlangrose commented Jan 28, 2026

Copy link
Copy Markdown
Member

Can you make the data converter its own dialog or dock widget? I think it is an addition to the modelling process rather than a first step as you can use the modelling without using the data conversion. Also do you have a demo dataset I can use to test it?

@lachlangrose

lachlangrose commented Jan 28, 2026•
edited by rabii-chaarani
Loading

Copy link
Copy Markdown
Member

@rabii-chaarani looks good, couple of easy things

  • use the column matcher from main.helper to automatically guess the appropriate file from the available files.
  • Restrict the layer types to the appropriate geometries e.g. geology should be polygon not line or point, fault/fold should be line and structure points.
  • Don't add xyz to the fault layer as they are a line layer, maybe add it as centre x, centre y, centre z.

Future idea

  • Add a logging capabilities to report to the user what was done by the tool. E.g. column x changed to column y, create a file from a layer etc.
  • Use the QGIS layers directly, not the underlying shapefile as the underlying shapefile is not compatible with filters.

CopilotAI reviewed Jan 30, 2026
View reviewed changes

CopilotAI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 13 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadloopstructural/main/m2l_api.py
Comment on lines 540 to 541
if unit_name_field != 'UNITNAME' and unit_name_field in geology_gdf.columns:
geology_gdf = geology_gdf.rename(columns={unit_name_field: 'UNITNAME'})

CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The unit name field is being renamed twice: once at line 492-494 and again at lines 540-541. This is redundant and could lead to confusion. The second rename should be removed as the first one already handles this transformation.

Suggested change
ifunit_name_field!='UNITNAME'andunit_name_fieldingeology_gdf.columns:
geology_gdf=geology_gdf.rename(columns={unit_name_field: 'UNITNAME'})

Copilot uses AI. Check for mistakes.
Comment threadloopstructural/main/m2l_api.py Outdated
)
sampled_contacts_gdf = qgsLayerToGeoDataFrame(sampled_contacts)
structure_gdf = qgsLayerToGeoDataFrame(structure)
structure_gdf = qgsLayerToGeoDataFrame(structure)

CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing whitespace found on line 460. This should be removed to maintain code cleanliness and consistency with the codebase style.

Copilot uses AI. Check for mistakes.
Comment threadloopstructural/gui/data_conversion/data_conversion_widget.py Outdated
Comment threadloopstructural/gui/map2loop_tools/thickness_calculator_widget.py
Comment threadloopstructural/gui/data_conversion/configuration.py Outdated
Comment threadloopstructural/gui/data_conversion/data_conversion_widget.py
if self.project is not None:
try:
combo.setProject(self.project)
except Exception:

CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change
exceptException:
exceptException:
# Some QGIS/Qt environments may not support setProject or may raise here;
# failure to bind the project is non-fatal, so we intentionally ignore it.

Copilot uses AI. Check for mistakes.
Comment threadloopstructural/main/vectorLayerWrapper.py
try:
data = bytes(data)
except Exception:
pass

CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change
pass
# Best-effort conversion to bytes; if this fails, fall back to using the
# original data and let the subsequent fromWkb call handle it.
logger.debug(
"Failed to convert WKB data to bytes in _geometry_from_value",
exc_info=True,
)

Copilot uses AI. Check for mistakes.
Comment threadloopstructural/main/vectorLayerWrapper.py
Comment on lines +657 to +671
except Exception:
pass
try:
epsg = crs_info.to_epsg()
if epsg:
return QgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
except Exception:
pass
if isinstance(crs_info, str):
try:
temp = QgsCoordinateReferenceSystem(crs_info)
if temp.isValid():
return temp
except Exception:
pass

CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change
exceptException:
pass
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptException:
pass
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptException:
pass
exceptExceptionase:
logger.debug("Failed to create CRS from WKT using createFromWkt: %r (%s)", text, e)
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptExceptionase:
logger.debug("Failed to derive EPSG from CRS info %r: %s", crs_info, e)
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptExceptionase:
logger.debug("Failed to construct CRS from string %r: %s", crs_info, e)

Copilot uses AI. Check for mistakes.
Comment threadloopstructural/main/vectorLayerWrapper.py
Comment on lines +657 to +671
except Exception:
pass
try:
epsg = crs_info.to_epsg()
if epsg:
return QgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
except Exception:
pass
if isinstance(crs_info, str):
try:
temp = QgsCoordinateReferenceSystem(crs_info)
if temp.isValid():
return temp
except Exception:
pass

CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change
exceptException:
pass
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptException:
pass
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptException:
pass
exceptExceptionasexc:
logger.debug(
"Failed to create CRS from WKT using legacy createFromWkt: %s",
exc,
)
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptExceptionasexc:
logger.debug(
"Failed to obtain EPSG code from CRS info %r: %s",
crs_info,
exc,
)
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptExceptionasexc:
logger.debug(
"Failed to construct CRS from string %r: %s",
crs_info,
exc,
)

Copilot uses AI. Check for mistakes.
Comment threadloopstructural/main/vectorLayerWrapper.py
elif isinstance(val, np.generic):
try:
val = val.item()
except Exception:

CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change
exceptException:
exceptException:
# If conversion to a native Python scalar fails, keep the original numpy scalar.

Copilot uses AI. Check for mistakes.
@lachlangrose
lachlangrose merged commit ea99e14 into dev-0.1.12Jan 30, 2026
1 check passed
@lachlangrose
lachlangrose deleted the data_converter_tool branch January 30, 2026 04:15
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@lachlangroselachlangroselachlangrose approved these changes

Assignees

No one assigned

Labels

qualityUI

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@rabii-chaarani@lachlangrose
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Data converter Widget by rabii-chaarani · Pull Request #76 · Loop3D/plugin_loopstructural · GitHub
Skip to content
Data converter Widget - #76
Merged
lachlangrose merged 20 commits into
dev-0.1.12from 
data_converter_tool
Jan 30, 2026
Merged
Data converter Widget#76
lachlangrose merged 20 commits into
dev-0.1.12from 
data_converter_tool
Conversation
@rabii-chaarani
@rabii-chaaranirabii-chaarani
commented
Jan 27, 2026

Copy link


Copy Markdown

Member
 No description provided. 
CopilotAI
reviewed
Jan 27, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
This pull request introduces a comprehensive "Data Converter Widget" that enables users to convert geological datasets for use within LoopStructural through both automatic and manual conversion workflows.
Changes:
  • Added a new data conversion widget with automatic and manual conversion modes
  • Implemented utility functions to convert GeoPandas GeoDataFrames and pandas DataFrames to QGIS vector layers
  • Integrated the data conversion widget into the main Loop widget interface
  • Added tests for the NTGS configuration model
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 17 comments.
Show a summary per file
FileDescription
tests/unit/gui/test_ntgs_configuration_model.pyUnit tests for configuration model (imports incorrect class names)
loopstructural/main/vectorLayerWrapper.pyNew functions to convert DataFrames/GeoDataFrames to QGIS layers
loopstructural/gui/loop_widget.pyIntegration of data conversion widget into main interface
loopstructural/gui/data_conversion/data_conversion_widget.pyMain widget implementation with automatic and manual conversion modes
loopstructural/gui/data_conversion/configuration.pyConfiguration state management for NTGS data types
loopstructural/gui/data_conversion/init.pyPackage initialization exposing DataConversionWidget
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadtests/unit/gui/test_ntgs_configuration_model.py
@@ -0,0 +1,33 @@
import pytest

from loopstructural.gui.data_conversion.configuration import NtgsConfig, NtgsConfigurationModel
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test imports NtgsConfig and NtgsConfigurationModel, but the actual configuration module defines these classes as Config and ConfigurationState. This import will fail at runtime. The imports should be updated to match the actual class names in loopstructural/gui/data_conversion/configuration.py.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+750
 to 
+764

 for _, row in df.iterrows():
 feat = QgsFeature(layer.fields())
 attrs = []
 for column in df.columns:
 val = row[column]
 if pd.isna(val):
 val = None
 elif isinstance(val, np.generic):
 try:
 val = val.item()
 except Exception:
 pass
 attrs.append(val)
 feat.setAttributes(attrs)
 features.append(feat)
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using iterrows() is generally inefficient for large DataFrames as it creates a new Series object for each row. For better performance with large datasets, consider using itertuples() or vectorized operations where possible.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/loop_widget.py
Outdated

 self.data_conversion_widget = DataConversionWidget(
 self,
 data_manager=self.data_manager,
 project=self.data_manager.project if self.data_manager else None,
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accessing self.data_manager.project could raise an AttributeError if data_manager is not None but lacks a project attribute. Consider using getattr(self.data_manager, 'project', None) for safer attribute access.

Suggested change

project=self.data_manager.projectifself.data_managerelseNone,
project=getattr(self.data_manager, "project", None)ifself.data_managerelseNone,
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/data_conversion/configuration.py
Outdated

Comment threadloopstructural/main/vectorLayerWrapper.py
 return layer


def QgsLayerFromDataFrame(dataframe, layer_name: str = "Converted Table"):
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function name QgsLayerFromDataFrame uses PascalCase, which is inconsistent with existing functions in this file like qgsLayerToGeoDataFrame, qgsLayerToDataFrame, and qgsRasterToGdalDataset that use camelCase. Consider renaming to qgsLayerFromDataFrame to maintain consistency with the existing naming convention in this file.

Suggested change

defQgsLayerFromDataFrame(dataframe, layer_name: str="Converted Table"):
defqgsLayerFromDataFrame(dataframe, layer_name: str="Converted Table"):
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 try:
 if temp.createFromWkt(text):
 return temp
 except Exception:
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
Comment threadloopstructural/main/vectorLayerWrapper.py
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# Ignore failures when interpreting CRS from string; fall back to default CRS.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+713
 to 
+714

 except Exception:
 pass
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
exceptExceptionasexc:
logger.debug("Failed to convert numpy scalar %r using .item(): %s", val, exc)
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+760
 to 
+761

 except Exception:
 pass
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
exceptExceptionasexc:
# Fall back to the original NumPy scalar if conversion fails.
logger.debug("Failed to convert NumPy scalar %r with .item(): %s", val, exc)
Copilot uses AI. Check for mistakes.

@lachlangrose
lachlangrose
commented
Jan 28, 2026

Copy link


Copy Markdown

Member
Can you make the data converter its own dialog or dock widget? I think it is an addition to the modelling process rather than a first step as you can use the modelling without using the data conversion. Also do you have a demo dataset I can use to test it?
@lachlangrose
lachlangrose
commented
Jan 28, 2026•

edited by rabii-chaarani

Loading

Copy link


Copy Markdown

Member
@rabii-chaarani looks good, couple of easy things
  •  use the column matcher from main.helper to automatically guess the appropriate file from the available files.
  •  Restrict the layer types to the appropriate geometries e.g. geology should be polygon not line or point, fault/fold should be line and structure points.
  •  Don't add xyz to the fault layer as they are a line layer, maybe add it as centre x, centre y, centre z.
Future idea
  •  Add a logging capabilities to report to the user what was done by the tool. E.g. column x changed to column y, create a file from a layer etc.
  •  Use the QGIS layers directly, not the underlying shapefile as the underlying shapefile is not compatible with filters.
CopilotAI
reviewed
Jan 30, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated 13 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadloopstructural/main/m2l_api.py

Comment on lines

540
 to 
541

 if unit_name_field != 'UNITNAME' and unit_name_field in geology_gdf.columns:
 geology_gdf = geology_gdf.rename(columns={unit_name_field: 'UNITNAME'})
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The unit name field is being renamed twice: once at line 492-494 and again at lines 540-541. This is redundant and could lead to confusion. The second rename should be removed as the first one already handles this transformation.

Suggested change

ifunit_name_field!='UNITNAME'andunit_name_fieldingeology_gdf.columns:
geology_gdf=geology_gdf.rename(columns={unit_name_field: 'UNITNAME'})
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/m2l_api.py
Outdated

 )
 sampled_contacts_gdf = qgsLayerToGeoDataFrame(sampled_contacts)
 structure_gdf = qgsLayerToGeoDataFrame(structure)
 structure_gdf = qgsLayerToGeoDataFrame(structure) 
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing whitespace found on line 460. This should be removed to maintain code cleanliness and consistency with the codebase style.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/data_conversion/data_conversion_widget.py
Outdated

Comment threadloopstructural/gui/map2loop_tools/thickness_calculator_widget.py
Comment threadloopstructural/gui/data_conversion/configuration.py
Outdated

Comment threadloopstructural/gui/data_conversion/data_conversion_widget.py
 if self.project is not None:
 try:
 combo.setProject(self.project)
 except Exception:
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# Some QGIS/Qt environments may not support setProject or may raise here;
# failure to bind the project is non-fatal, so we intentionally ignore it.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 try:
 data = bytes(data)
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

pass
# Best-effort conversion to bytes; if this fails, fall back to using the
# original data and let the subsequent fromWkb call handle it.
logger.debug(
"Failed to convert WKB data to bytes in _geometry_from_value",
exc_info=True,
 )
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+657
 to 
+671

 except Exception:
 pass
 try:
 epsg = crs_info.to_epsg()
 if epsg:
 return QgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
 except Exception:
 pass
 if isinstance(crs_info, str):
 try:
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptException:
pass
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptException:
pass
exceptExceptionase:
logger.debug("Failed to create CRS from WKT using createFromWkt: %r (%s)", text, e)
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptExceptionase:
logger.debug("Failed to derive EPSG from CRS info %r: %s", crs_info, e)
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptExceptionase:
logger.debug("Failed to construct CRS from string %r: %s", crs_info, e)
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+657
 to 
+671

 except Exception:
 pass
 try:
 epsg = crs_info.to_epsg()
 if epsg:
 return QgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
 except Exception:
 pass
 if isinstance(crs_info, str):
 try:
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptException:
pass
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptException:
pass
exceptExceptionasexc:
logger.debug(
"Failed to create CRS from WKT using legacy createFromWkt: %s",
exc,
 )
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptExceptionasexc:
logger.debug(
"Failed to obtain EPSG code from CRS info %r: %s",
crs_info,
exc,
 )
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptExceptionasexc:
logger.debug(
"Failed to construct CRS from string %r: %s",
crs_info,
exc,
 )
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 elif isinstance(val, np.generic):
 try:
 val = val.item()
 except Exception:
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# If conversion to a native Python scalar fails, keep the original numpy scalar.
Copilot uses AI. Check for mistakes.

@lachlangrose
lachlangrose
merged commit ea99e14
into
dev-0.1.12Jan 30, 2026

1 check passed

@lachlangrose
lachlangrose
deleted the

data_converter_tool

branch
January 30, 2026 04:15
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@lachlangroselachlangroselachlangrose approved these changes

Assignees


No one assigned


Labels

qualityUI

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@rabii-chaarani@lachlangrose

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Data converter Widget by rabii-chaarani · Pull Request #76 · Loop3D/plugin_loopstructural · GitHub
Skip to content
Data converter Widget - #76
Merged
lachlangrose merged 20 commits into
dev-0.1.12from 
data_converter_tool
Jan 30, 2026
Merged
Data converter Widget#76
lachlangrose merged 20 commits into
dev-0.1.12from 
data_converter_tool
Conversation
@rabii-chaarani
@rabii-chaaranirabii-chaarani
commented
Jan 27, 2026

Copy link


Copy Markdown

Member
 No description provided. 
CopilotAI
reviewed
Jan 27, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
This pull request introduces a comprehensive "Data Converter Widget" that enables users to convert geological datasets for use within LoopStructural through both automatic and manual conversion workflows.
Changes:
  • Added a new data conversion widget with automatic and manual conversion modes
  • Implemented utility functions to convert GeoPandas GeoDataFrames and pandas DataFrames to QGIS vector layers
  • Integrated the data conversion widget into the main Loop widget interface
  • Added tests for the NTGS configuration model
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 17 comments.
Show a summary per file
FileDescription
tests/unit/gui/test_ntgs_configuration_model.pyUnit tests for configuration model (imports incorrect class names)
loopstructural/main/vectorLayerWrapper.pyNew functions to convert DataFrames/GeoDataFrames to QGIS layers
loopstructural/gui/loop_widget.pyIntegration of data conversion widget into main interface
loopstructural/gui/data_conversion/data_conversion_widget.pyMain widget implementation with automatic and manual conversion modes
loopstructural/gui/data_conversion/configuration.pyConfiguration state management for NTGS data types
loopstructural/gui/data_conversion/init.pyPackage initialization exposing DataConversionWidget
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadtests/unit/gui/test_ntgs_configuration_model.py
@@ -0,0 +1,33 @@
import pytest

from loopstructural.gui.data_conversion.configuration import NtgsConfig, NtgsConfigurationModel
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test imports NtgsConfig and NtgsConfigurationModel, but the actual configuration module defines these classes as Config and ConfigurationState. This import will fail at runtime. The imports should be updated to match the actual class names in loopstructural/gui/data_conversion/configuration.py.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+750
 to 
+764

 for _, row in df.iterrows():
 feat = QgsFeature(layer.fields())
 attrs = []
 for column in df.columns:
 val = row[column]
 if pd.isna(val):
 val = None
 elif isinstance(val, np.generic):
 try:
 val = val.item()
 except Exception:
 pass
 attrs.append(val)
 feat.setAttributes(attrs)
 features.append(feat)
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using iterrows() is generally inefficient for large DataFrames as it creates a new Series object for each row. For better performance with large datasets, consider using itertuples() or vectorized operations where possible.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/loop_widget.py
Outdated

 self.data_conversion_widget = DataConversionWidget(
 self,
 data_manager=self.data_manager,
 project=self.data_manager.project if self.data_manager else None,
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accessing self.data_manager.project could raise an AttributeError if data_manager is not None but lacks a project attribute. Consider using getattr(self.data_manager, 'project', None) for safer attribute access.

Suggested change

project=self.data_manager.projectifself.data_managerelseNone,
project=getattr(self.data_manager, "project", None)ifself.data_managerelseNone,
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/data_conversion/configuration.py
Outdated

Comment threadloopstructural/main/vectorLayerWrapper.py
 return layer


def QgsLayerFromDataFrame(dataframe, layer_name: str = "Converted Table"):
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function name QgsLayerFromDataFrame uses PascalCase, which is inconsistent with existing functions in this file like qgsLayerToGeoDataFrame, qgsLayerToDataFrame, and qgsRasterToGdalDataset that use camelCase. Consider renaming to qgsLayerFromDataFrame to maintain consistency with the existing naming convention in this file.

Suggested change

defQgsLayerFromDataFrame(dataframe, layer_name: str="Converted Table"):
defqgsLayerFromDataFrame(dataframe, layer_name: str="Converted Table"):
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 try:
 if temp.createFromWkt(text):
 return temp
 except Exception:
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
Comment threadloopstructural/main/vectorLayerWrapper.py
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# Ignore failures when interpreting CRS from string; fall back to default CRS.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+713
 to 
+714

 except Exception:
 pass
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
exceptExceptionasexc:
logger.debug("Failed to convert numpy scalar %r using .item(): %s", val, exc)
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+760
 to 
+761

 except Exception:
 pass
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
exceptExceptionasexc:
# Fall back to the original NumPy scalar if conversion fails.
logger.debug("Failed to convert NumPy scalar %r with .item(): %s", val, exc)
Copilot uses AI. Check for mistakes.

@lachlangrose
lachlangrose
commented
Jan 28, 2026

Copy link


Copy Markdown

Member
Can you make the data converter its own dialog or dock widget? I think it is an addition to the modelling process rather than a first step as you can use the modelling without using the data conversion. Also do you have a demo dataset I can use to test it?
@lachlangrose
lachlangrose
commented
Jan 28, 2026•

edited by rabii-chaarani

Loading

Copy link


Copy Markdown

Member
@rabii-chaarani looks good, couple of easy things
  •  use the column matcher from main.helper to automatically guess the appropriate file from the available files.
  •  Restrict the layer types to the appropriate geometries e.g. geology should be polygon not line or point, fault/fold should be line and structure points.
  •  Don't add xyz to the fault layer as they are a line layer, maybe add it as centre x, centre y, centre z.
Future idea
  •  Add a logging capabilities to report to the user what was done by the tool. E.g. column x changed to column y, create a file from a layer etc.
  •  Use the QGIS layers directly, not the underlying shapefile as the underlying shapefile is not compatible with filters.
CopilotAI
reviewed
Jan 30, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated 13 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadloopstructural/main/m2l_api.py

Comment on lines

540
 to 
541

 if unit_name_field != 'UNITNAME' and unit_name_field in geology_gdf.columns:
 geology_gdf = geology_gdf.rename(columns={unit_name_field: 'UNITNAME'})
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The unit name field is being renamed twice: once at line 492-494 and again at lines 540-541. This is redundant and could lead to confusion. The second rename should be removed as the first one already handles this transformation.

Suggested change

ifunit_name_field!='UNITNAME'andunit_name_fieldingeology_gdf.columns:
geology_gdf=geology_gdf.rename(columns={unit_name_field: 'UNITNAME'})
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/m2l_api.py
Outdated

 )
 sampled_contacts_gdf = qgsLayerToGeoDataFrame(sampled_contacts)
 structure_gdf = qgsLayerToGeoDataFrame(structure)
 structure_gdf = qgsLayerToGeoDataFrame(structure) 
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing whitespace found on line 460. This should be removed to maintain code cleanliness and consistency with the codebase style.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/data_conversion/data_conversion_widget.py
Outdated

Comment threadloopstructural/gui/map2loop_tools/thickness_calculator_widget.py
Comment threadloopstructural/gui/data_conversion/configuration.py
Outdated

Comment threadloopstructural/gui/data_conversion/data_conversion_widget.py
 if self.project is not None:
 try:
 combo.setProject(self.project)
 except Exception:
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# Some QGIS/Qt environments may not support setProject or may raise here;
# failure to bind the project is non-fatal, so we intentionally ignore it.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 try:
 data = bytes(data)
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

pass
# Best-effort conversion to bytes; if this fails, fall back to using the
# original data and let the subsequent fromWkb call handle it.
logger.debug(
"Failed to convert WKB data to bytes in _geometry_from_value",
exc_info=True,
 )
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+657
 to 
+671

 except Exception:
 pass
 try:
 epsg = crs_info.to_epsg()
 if epsg:
 return QgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
 except Exception:
 pass
 if isinstance(crs_info, str):
 try:
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptException:
pass
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptException:
pass
exceptExceptionase:
logger.debug("Failed to create CRS from WKT using createFromWkt: %r (%s)", text, e)
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptExceptionase:
logger.debug("Failed to derive EPSG from CRS info %r: %s", crs_info, e)
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptExceptionase:
logger.debug("Failed to construct CRS from string %r: %s", crs_info, e)
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+657
 to 
+671

 except Exception:
 pass
 try:
 epsg = crs_info.to_epsg()
 if epsg:
 return QgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
 except Exception:
 pass
 if isinstance(crs_info, str):
 try:
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptException:
pass
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptException:
pass
exceptExceptionasexc:
logger.debug(
"Failed to create CRS from WKT using legacy createFromWkt: %s",
exc,
 )
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptExceptionasexc:
logger.debug(
"Failed to obtain EPSG code from CRS info %r: %s",
crs_info,
exc,
 )
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptExceptionasexc:
logger.debug(
"Failed to construct CRS from string %r: %s",
crs_info,
exc,
 )
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 elif isinstance(val, np.generic):
 try:
 val = val.item()
 except Exception:
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# If conversion to a native Python scalar fails, keep the original numpy scalar.
Copilot uses AI. Check for mistakes.

@lachlangrose
lachlangrose
merged commit ea99e14
into
dev-0.1.12Jan 30, 2026

1 check passed

@lachlangrose
lachlangrose
deleted the

data_converter_tool

branch
January 30, 2026 04:15
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@lachlangroselachlangroselachlangrose approved these changes

Assignees


No one assigned


Labels

qualityUI

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@rabii-chaarani@lachlangrose

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Data converter Widget by rabii-chaarani · Pull Request #76 · Loop3D/plugin_loopstructural · GitHub
Skip to content
Data converter Widget - #76
Merged
lachlangrose merged 20 commits into
dev-0.1.12from 
data_converter_tool
Jan 30, 2026
Merged
Data converter Widget#76
lachlangrose merged 20 commits into
dev-0.1.12from 
data_converter_tool
Conversation
@rabii-chaarani
@rabii-chaaranirabii-chaarani
commented
Jan 27, 2026

Copy link


Copy Markdown

Member
 No description provided. 
CopilotAI
reviewed
Jan 27, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
This pull request introduces a comprehensive "Data Converter Widget" that enables users to convert geological datasets for use within LoopStructural through both automatic and manual conversion workflows.
Changes:
  • Added a new data conversion widget with automatic and manual conversion modes
  • Implemented utility functions to convert GeoPandas GeoDataFrames and pandas DataFrames to QGIS vector layers
  • Integrated the data conversion widget into the main Loop widget interface
  • Added tests for the NTGS configuration model
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 17 comments.
Show a summary per file
FileDescription
tests/unit/gui/test_ntgs_configuration_model.pyUnit tests for configuration model (imports incorrect class names)
loopstructural/main/vectorLayerWrapper.pyNew functions to convert DataFrames/GeoDataFrames to QGIS layers
loopstructural/gui/loop_widget.pyIntegration of data conversion widget into main interface
loopstructural/gui/data_conversion/data_conversion_widget.pyMain widget implementation with automatic and manual conversion modes
loopstructural/gui/data_conversion/configuration.pyConfiguration state management for NTGS data types
loopstructural/gui/data_conversion/init.pyPackage initialization exposing DataConversionWidget
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadtests/unit/gui/test_ntgs_configuration_model.py
@@ -0,0 +1,33 @@
import pytest

from loopstructural.gui.data_conversion.configuration import NtgsConfig, NtgsConfigurationModel
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test imports NtgsConfig and NtgsConfigurationModel, but the actual configuration module defines these classes as Config and ConfigurationState. This import will fail at runtime. The imports should be updated to match the actual class names in loopstructural/gui/data_conversion/configuration.py.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+750
 to 
+764

 for _, row in df.iterrows():
 feat = QgsFeature(layer.fields())
 attrs = []
 for column in df.columns:
 val = row[column]
 if pd.isna(val):
 val = None
 elif isinstance(val, np.generic):
 try:
 val = val.item()
 except Exception:
 pass
 attrs.append(val)
 feat.setAttributes(attrs)
 features.append(feat)
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using iterrows() is generally inefficient for large DataFrames as it creates a new Series object for each row. For better performance with large datasets, consider using itertuples() or vectorized operations where possible.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/loop_widget.py
Outdated

 self.data_conversion_widget = DataConversionWidget(
 self,
 data_manager=self.data_manager,
 project=self.data_manager.project if self.data_manager else None,
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accessing self.data_manager.project could raise an AttributeError if data_manager is not None but lacks a project attribute. Consider using getattr(self.data_manager, 'project', None) for safer attribute access.

Suggested change

project=self.data_manager.projectifself.data_managerelseNone,
project=getattr(self.data_manager, "project", None)ifself.data_managerelseNone,
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/data_conversion/configuration.py
Outdated

Comment threadloopstructural/main/vectorLayerWrapper.py
 return layer


def QgsLayerFromDataFrame(dataframe, layer_name: str = "Converted Table"):
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function name QgsLayerFromDataFrame uses PascalCase, which is inconsistent with existing functions in this file like qgsLayerToGeoDataFrame, qgsLayerToDataFrame, and qgsRasterToGdalDataset that use camelCase. Consider renaming to qgsLayerFromDataFrame to maintain consistency with the existing naming convention in this file.

Suggested change

defQgsLayerFromDataFrame(dataframe, layer_name: str="Converted Table"):
defqgsLayerFromDataFrame(dataframe, layer_name: str="Converted Table"):
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 try:
 if temp.createFromWkt(text):
 return temp
 except Exception:
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
Comment threadloopstructural/main/vectorLayerWrapper.py
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# Ignore failures when interpreting CRS from string; fall back to default CRS.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+713
 to 
+714

 except Exception:
 pass
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
exceptExceptionasexc:
logger.debug("Failed to convert numpy scalar %r using .item(): %s", val, exc)
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+760
 to 
+761

 except Exception:
 pass
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
exceptExceptionasexc:
# Fall back to the original NumPy scalar if conversion fails.
logger.debug("Failed to convert NumPy scalar %r with .item(): %s", val, exc)
Copilot uses AI. Check for mistakes.

@lachlangrose
lachlangrose
commented
Jan 28, 2026

Copy link


Copy Markdown

Member
Can you make the data converter its own dialog or dock widget? I think it is an addition to the modelling process rather than a first step as you can use the modelling without using the data conversion. Also do you have a demo dataset I can use to test it?
@lachlangrose
lachlangrose
commented
Jan 28, 2026•

edited by rabii-chaarani

Loading

Copy link


Copy Markdown

Member
@rabii-chaarani looks good, couple of easy things
  •  use the column matcher from main.helper to automatically guess the appropriate file from the available files.
  •  Restrict the layer types to the appropriate geometries e.g. geology should be polygon not line or point, fault/fold should be line and structure points.
  •  Don't add xyz to the fault layer as they are a line layer, maybe add it as centre x, centre y, centre z.
Future idea
  •  Add a logging capabilities to report to the user what was done by the tool. E.g. column x changed to column y, create a file from a layer etc.
  •  Use the QGIS layers directly, not the underlying shapefile as the underlying shapefile is not compatible with filters.
CopilotAI
reviewed
Jan 30, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated 13 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadloopstructural/main/m2l_api.py

Comment on lines

540
 to 
541

 if unit_name_field != 'UNITNAME' and unit_name_field in geology_gdf.columns:
 geology_gdf = geology_gdf.rename(columns={unit_name_field: 'UNITNAME'})
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The unit name field is being renamed twice: once at line 492-494 and again at lines 540-541. This is redundant and could lead to confusion. The second rename should be removed as the first one already handles this transformation.

Suggested change

ifunit_name_field!='UNITNAME'andunit_name_fieldingeology_gdf.columns:
geology_gdf=geology_gdf.rename(columns={unit_name_field: 'UNITNAME'})
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/m2l_api.py
Outdated

 )
 sampled_contacts_gdf = qgsLayerToGeoDataFrame(sampled_contacts)
 structure_gdf = qgsLayerToGeoDataFrame(structure)
 structure_gdf = qgsLayerToGeoDataFrame(structure) 
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing whitespace found on line 460. This should be removed to maintain code cleanliness and consistency with the codebase style.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/data_conversion/data_conversion_widget.py
Outdated

Comment threadloopstructural/gui/map2loop_tools/thickness_calculator_widget.py
Comment threadloopstructural/gui/data_conversion/configuration.py
Outdated

Comment threadloopstructural/gui/data_conversion/data_conversion_widget.py
 if self.project is not None:
 try:
 combo.setProject(self.project)
 except Exception:
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# Some QGIS/Qt environments may not support setProject or may raise here;
# failure to bind the project is non-fatal, so we intentionally ignore it.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 try:
 data = bytes(data)
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

pass
# Best-effort conversion to bytes; if this fails, fall back to using the
# original data and let the subsequent fromWkb call handle it.
logger.debug(
"Failed to convert WKB data to bytes in _geometry_from_value",
exc_info=True,
 )
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+657
 to 
+671

 except Exception:
 pass
 try:
 epsg = crs_info.to_epsg()
 if epsg:
 return QgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
 except Exception:
 pass
 if isinstance(crs_info, str):
 try:
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptException:
pass
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptException:
pass
exceptExceptionase:
logger.debug("Failed to create CRS from WKT using createFromWkt: %r (%s)", text, e)
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptExceptionase:
logger.debug("Failed to derive EPSG from CRS info %r: %s", crs_info, e)
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptExceptionase:
logger.debug("Failed to construct CRS from string %r: %s", crs_info, e)
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+657
 to 
+671

 except Exception:
 pass
 try:
 epsg = crs_info.to_epsg()
 if epsg:
 return QgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
 except Exception:
 pass
 if isinstance(crs_info, str):
 try:
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptException:
pass
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptException:
pass
exceptExceptionasexc:
logger.debug(
"Failed to create CRS from WKT using legacy createFromWkt: %s",
exc,
 )
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptExceptionasexc:
logger.debug(
"Failed to obtain EPSG code from CRS info %r: %s",
crs_info,
exc,
 )
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptExceptionasexc:
logger.debug(
"Failed to construct CRS from string %r: %s",
crs_info,
exc,
 )
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 elif isinstance(val, np.generic):
 try:
 val = val.item()
 except Exception:
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# If conversion to a native Python scalar fails, keep the original numpy scalar.
Copilot uses AI. Check for mistakes.

@lachlangrose
lachlangrose
merged commit ea99e14
into
dev-0.1.12Jan 30, 2026

1 check passed

@lachlangrose
lachlangrose
deleted the

data_converter_tool

branch
January 30, 2026 04:15
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@lachlangroselachlangroselachlangrose approved these changes

Assignees


No one assigned


Labels

qualityUI

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@rabii-chaarani@lachlangrose

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
Data converter Widget by rabii-chaarani · Pull Request #76 · Loop3D/plugin_loopstructural · GitHub
Skip to content
Data converter Widget - #76
Merged
lachlangrose merged 20 commits into
dev-0.1.12from 
data_converter_tool
Jan 30, 2026
Merged
Data converter Widget#76
lachlangrose merged 20 commits into
dev-0.1.12from 
data_converter_tool
Conversation
@rabii-chaarani
@rabii-chaaranirabii-chaarani
commented
Jan 27, 2026

Copy link


Copy Markdown

Member
 No description provided. 
CopilotAI
reviewed
Jan 27, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
This pull request introduces a comprehensive "Data Converter Widget" that enables users to convert geological datasets for use within LoopStructural through both automatic and manual conversion workflows.
Changes:
  • Added a new data conversion widget with automatic and manual conversion modes
  • Implemented utility functions to convert GeoPandas GeoDataFrames and pandas DataFrames to QGIS vector layers
  • Integrated the data conversion widget into the main Loop widget interface
  • Added tests for the NTGS configuration model
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 17 comments.
Show a summary per file
FileDescription
tests/unit/gui/test_ntgs_configuration_model.pyUnit tests for configuration model (imports incorrect class names)
loopstructural/main/vectorLayerWrapper.pyNew functions to convert DataFrames/GeoDataFrames to QGIS layers
loopstructural/gui/loop_widget.pyIntegration of data conversion widget into main interface
loopstructural/gui/data_conversion/data_conversion_widget.pyMain widget implementation with automatic and manual conversion modes
loopstructural/gui/data_conversion/configuration.pyConfiguration state management for NTGS data types
loopstructural/gui/data_conversion/init.pyPackage initialization exposing DataConversionWidget
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadtests/unit/gui/test_ntgs_configuration_model.py
@@ -0,0 +1,33 @@
import pytest

from loopstructural.gui.data_conversion.configuration import NtgsConfig, NtgsConfigurationModel
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test imports NtgsConfig and NtgsConfigurationModel, but the actual configuration module defines these classes as Config and ConfigurationState. This import will fail at runtime. The imports should be updated to match the actual class names in loopstructural/gui/data_conversion/configuration.py.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+750
 to 
+764

 for _, row in df.iterrows():
 feat = QgsFeature(layer.fields())
 attrs = []
 for column in df.columns:
 val = row[column]
 if pd.isna(val):
 val = None
 elif isinstance(val, np.generic):
 try:
 val = val.item()
 except Exception:
 pass
 attrs.append(val)
 feat.setAttributes(attrs)
 features.append(feat)
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using iterrows() is generally inefficient for large DataFrames as it creates a new Series object for each row. For better performance with large datasets, consider using itertuples() or vectorized operations where possible.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/loop_widget.py
Outdated

 self.data_conversion_widget = DataConversionWidget(
 self,
 data_manager=self.data_manager,
 project=self.data_manager.project if self.data_manager else None,
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accessing self.data_manager.project could raise an AttributeError if data_manager is not None but lacks a project attribute. Consider using getattr(self.data_manager, 'project', None) for safer attribute access.

Suggested change

project=self.data_manager.projectifself.data_managerelseNone,
project=getattr(self.data_manager, "project", None)ifself.data_managerelseNone,
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/data_conversion/configuration.py
Outdated

Comment threadloopstructural/main/vectorLayerWrapper.py
 return layer


def QgsLayerFromDataFrame(dataframe, layer_name: str = "Converted Table"):
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function name QgsLayerFromDataFrame uses PascalCase, which is inconsistent with existing functions in this file like qgsLayerToGeoDataFrame, qgsLayerToDataFrame, and qgsRasterToGdalDataset that use camelCase. Consider renaming to qgsLayerFromDataFrame to maintain consistency with the existing naming convention in this file.

Suggested change

defQgsLayerFromDataFrame(dataframe, layer_name: str="Converted Table"):
defqgsLayerFromDataFrame(dataframe, layer_name: str="Converted Table"):
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 try:
 if temp.createFromWkt(text):
 return temp
 except Exception:
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
Comment threadloopstructural/main/vectorLayerWrapper.py
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# Ignore failures when interpreting CRS from string; fall back to default CRS.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+713
 to 
+714

 except Exception:
 pass
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
exceptExceptionasexc:
logger.debug("Failed to convert numpy scalar %r using .item(): %s", val, exc)
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+760
 to 
+761

 except Exception:
 pass
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
exceptExceptionasexc:
# Fall back to the original NumPy scalar if conversion fails.
logger.debug("Failed to convert NumPy scalar %r with .item(): %s", val, exc)
Copilot uses AI. Check for mistakes.

@lachlangrose
lachlangrose
commented
Jan 28, 2026

Copy link


Copy Markdown

Member
Can you make the data converter its own dialog or dock widget? I think it is an addition to the modelling process rather than a first step as you can use the modelling without using the data conversion. Also do you have a demo dataset I can use to test it?
@lachlangrose
lachlangrose
commented
Jan 28, 2026•

edited by rabii-chaarani

Loading

Copy link


Copy Markdown

Member
@rabii-chaarani looks good, couple of easy things
  •  use the column matcher from main.helper to automatically guess the appropriate file from the available files.
  •  Restrict the layer types to the appropriate geometries e.g. geology should be polygon not line or point, fault/fold should be line and structure points.
  •  Don't add xyz to the fault layer as they are a line layer, maybe add it as centre x, centre y, centre z.
Future idea
  •  Add a logging capabilities to report to the user what was done by the tool. E.g. column x changed to column y, create a file from a layer etc.
  •  Use the QGIS layers directly, not the underlying shapefile as the underlying shapefile is not compatible with filters.
CopilotAI
reviewed
Jan 30, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated 13 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadloopstructural/main/m2l_api.py

Comment on lines

540
 to 
541

 if unit_name_field != 'UNITNAME' and unit_name_field in geology_gdf.columns:
 geology_gdf = geology_gdf.rename(columns={unit_name_field: 'UNITNAME'})
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The unit name field is being renamed twice: once at line 492-494 and again at lines 540-541. This is redundant and could lead to confusion. The second rename should be removed as the first one already handles this transformation.

Suggested change

ifunit_name_field!='UNITNAME'andunit_name_fieldingeology_gdf.columns:
geology_gdf=geology_gdf.rename(columns={unit_name_field: 'UNITNAME'})
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/m2l_api.py
Outdated

 )
 sampled_contacts_gdf = qgsLayerToGeoDataFrame(sampled_contacts)
 structure_gdf = qgsLayerToGeoDataFrame(structure)
 structure_gdf = qgsLayerToGeoDataFrame(structure) 
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing whitespace found on line 460. This should be removed to maintain code cleanliness and consistency with the codebase style.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/data_conversion/data_conversion_widget.py
Outdated

Comment threadloopstructural/gui/map2loop_tools/thickness_calculator_widget.py
Comment threadloopstructural/gui/data_conversion/configuration.py
Outdated

Comment threadloopstructural/gui/data_conversion/data_conversion_widget.py
 if self.project is not None:
 try:
 combo.setProject(self.project)
 except Exception:
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# Some QGIS/Qt environments may not support setProject or may raise here;
# failure to bind the project is non-fatal, so we intentionally ignore it.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 try:
 data = bytes(data)
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

pass
# Best-effort conversion to bytes; if this fails, fall back to using the
# original data and let the subsequent fromWkb call handle it.
logger.debug(
"Failed to convert WKB data to bytes in _geometry_from_value",
exc_info=True,
 )
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+657
 to 
+671

 except Exception:
 pass
 try:
 epsg = crs_info.to_epsg()
 if epsg:
 return QgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
 except Exception:
 pass
 if isinstance(crs_info, str):
 try:
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptException:
pass
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptException:
pass
exceptExceptionase:
logger.debug("Failed to create CRS from WKT using createFromWkt: %r (%s)", text, e)
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptExceptionase:
logger.debug("Failed to derive EPSG from CRS info %r: %s", crs_info, e)
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptExceptionase:
logger.debug("Failed to construct CRS from string %r: %s", crs_info, e)
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+657
 to 
+671

 except Exception:
 pass
 try:
 epsg = crs_info.to_epsg()
 if epsg:
 return QgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
 except Exception:
 pass
 if isinstance(crs_info, str):
 try:
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptException:
pass
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptException:
pass
exceptExceptionasexc:
logger.debug(
"Failed to create CRS from WKT using legacy createFromWkt: %s",
exc,
 )
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptExceptionasexc:
logger.debug(
"Failed to obtain EPSG code from CRS info %r: %s",
crs_info,
exc,
 )
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptExceptionasexc:
logger.debug(
"Failed to construct CRS from string %r: %s",
crs_info,
exc,
 )
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 elif isinstance(val, np.generic):
 try:
 val = val.item()
 except Exception:
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# If conversion to a native Python scalar fails, keep the original numpy scalar.
Copilot uses AI. Check for mistakes.

@lachlangrose
lachlangrose
merged commit ea99e14
into
dev-0.1.12Jan 30, 2026

1 check passed

@lachlangrose
lachlangrose
deleted the

data_converter_tool

branch
January 30, 2026 04:15
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@lachlangroselachlangroselachlangrose approved these changes

Assignees


No one assigned


Labels

qualityUI

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@rabii-chaarani@lachlangrose

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Data converter Widget by rabii-chaarani · Pull Request #76 · Loop3D/plugin_loopstructural · GitHub
Skip to content
Data converter Widget - #76
Merged
lachlangrose merged 20 commits into
dev-0.1.12from 
data_converter_tool
Jan 30, 2026
Merged
Data converter Widget#76
lachlangrose merged 20 commits into
dev-0.1.12from 
data_converter_tool
Conversation
@rabii-chaarani
@rabii-chaaranirabii-chaarani
commented
Jan 27, 2026

Copy link


Copy Markdown

Member
 No description provided. 
CopilotAI
reviewed
Jan 27, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
This pull request introduces a comprehensive "Data Converter Widget" that enables users to convert geological datasets for use within LoopStructural through both automatic and manual conversion workflows.
Changes:
  • Added a new data conversion widget with automatic and manual conversion modes
  • Implemented utility functions to convert GeoPandas GeoDataFrames and pandas DataFrames to QGIS vector layers
  • Integrated the data conversion widget into the main Loop widget interface
  • Added tests for the NTGS configuration model
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 17 comments.
Show a summary per file
FileDescription
tests/unit/gui/test_ntgs_configuration_model.pyUnit tests for configuration model (imports incorrect class names)
loopstructural/main/vectorLayerWrapper.pyNew functions to convert DataFrames/GeoDataFrames to QGIS layers
loopstructural/gui/loop_widget.pyIntegration of data conversion widget into main interface
loopstructural/gui/data_conversion/data_conversion_widget.pyMain widget implementation with automatic and manual conversion modes
loopstructural/gui/data_conversion/configuration.pyConfiguration state management for NTGS data types
loopstructural/gui/data_conversion/init.pyPackage initialization exposing DataConversionWidget
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadtests/unit/gui/test_ntgs_configuration_model.py
@@ -0,0 +1,33 @@
import pytest

from loopstructural.gui.data_conversion.configuration import NtgsConfig, NtgsConfigurationModel
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test imports NtgsConfig and NtgsConfigurationModel, but the actual configuration module defines these classes as Config and ConfigurationState. This import will fail at runtime. The imports should be updated to match the actual class names in loopstructural/gui/data_conversion/configuration.py.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+750
 to 
+764

 for _, row in df.iterrows():
 feat = QgsFeature(layer.fields())
 attrs = []
 for column in df.columns:
 val = row[column]
 if pd.isna(val):
 val = None
 elif isinstance(val, np.generic):
 try:
 val = val.item()
 except Exception:
 pass
 attrs.append(val)
 feat.setAttributes(attrs)
 features.append(feat)
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using iterrows() is generally inefficient for large DataFrames as it creates a new Series object for each row. For better performance with large datasets, consider using itertuples() or vectorized operations where possible.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/loop_widget.py
Outdated

 self.data_conversion_widget = DataConversionWidget(
 self,
 data_manager=self.data_manager,
 project=self.data_manager.project if self.data_manager else None,
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accessing self.data_manager.project could raise an AttributeError if data_manager is not None but lacks a project attribute. Consider using getattr(self.data_manager, 'project', None) for safer attribute access.

Suggested change

project=self.data_manager.projectifself.data_managerelseNone,
project=getattr(self.data_manager, "project", None)ifself.data_managerelseNone,
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/data_conversion/configuration.py
Outdated

Comment threadloopstructural/main/vectorLayerWrapper.py
 return layer


def QgsLayerFromDataFrame(dataframe, layer_name: str = "Converted Table"):
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function name QgsLayerFromDataFrame uses PascalCase, which is inconsistent with existing functions in this file like qgsLayerToGeoDataFrame, qgsLayerToDataFrame, and qgsRasterToGdalDataset that use camelCase. Consider renaming to qgsLayerFromDataFrame to maintain consistency with the existing naming convention in this file.

Suggested change

defQgsLayerFromDataFrame(dataframe, layer_name: str="Converted Table"):
defqgsLayerFromDataFrame(dataframe, layer_name: str="Converted Table"):
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 try:
 if temp.createFromWkt(text):
 return temp
 except Exception:
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
Comment threadloopstructural/main/vectorLayerWrapper.py
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# Ignore failures when interpreting CRS from string; fall back to default CRS.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+713
 to 
+714

 except Exception:
 pass
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
exceptExceptionasexc:
logger.debug("Failed to convert numpy scalar %r using .item(): %s", val, exc)
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+760
 to 
+761

 except Exception:
 pass
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
exceptExceptionasexc:
# Fall back to the original NumPy scalar if conversion fails.
logger.debug("Failed to convert NumPy scalar %r with .item(): %s", val, exc)
Copilot uses AI. Check for mistakes.

@lachlangrose
lachlangrose
commented
Jan 28, 2026

Copy link


Copy Markdown

Member
Can you make the data converter its own dialog or dock widget? I think it is an addition to the modelling process rather than a first step as you can use the modelling without using the data conversion. Also do you have a demo dataset I can use to test it?
@lachlangrose
lachlangrose
commented
Jan 28, 2026•

edited by rabii-chaarani

Loading

Copy link


Copy Markdown

Member
@rabii-chaarani looks good, couple of easy things
  •  use the column matcher from main.helper to automatically guess the appropriate file from the available files.
  •  Restrict the layer types to the appropriate geometries e.g. geology should be polygon not line or point, fault/fold should be line and structure points.
  •  Don't add xyz to the fault layer as they are a line layer, maybe add it as centre x, centre y, centre z.
Future idea
  •  Add a logging capabilities to report to the user what was done by the tool. E.g. column x changed to column y, create a file from a layer etc.
  •  Use the QGIS layers directly, not the underlying shapefile as the underlying shapefile is not compatible with filters.
CopilotAI
reviewed
Jan 30, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated 13 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadloopstructural/main/m2l_api.py

Comment on lines

540
 to 
541

 if unit_name_field != 'UNITNAME' and unit_name_field in geology_gdf.columns:
 geology_gdf = geology_gdf.rename(columns={unit_name_field: 'UNITNAME'})
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The unit name field is being renamed twice: once at line 492-494 and again at lines 540-541. This is redundant and could lead to confusion. The second rename should be removed as the first one already handles this transformation.

Suggested change

ifunit_name_field!='UNITNAME'andunit_name_fieldingeology_gdf.columns:
geology_gdf=geology_gdf.rename(columns={unit_name_field: 'UNITNAME'})
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/m2l_api.py
Outdated

 )
 sampled_contacts_gdf = qgsLayerToGeoDataFrame(sampled_contacts)
 structure_gdf = qgsLayerToGeoDataFrame(structure)
 structure_gdf = qgsLayerToGeoDataFrame(structure) 
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing whitespace found on line 460. This should be removed to maintain code cleanliness and consistency with the codebase style.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/data_conversion/data_conversion_widget.py
Outdated

Comment threadloopstructural/gui/map2loop_tools/thickness_calculator_widget.py
Comment threadloopstructural/gui/data_conversion/configuration.py
Outdated

Comment threadloopstructural/gui/data_conversion/data_conversion_widget.py
 if self.project is not None:
 try:
 combo.setProject(self.project)
 except Exception:
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# Some QGIS/Qt environments may not support setProject or may raise here;
# failure to bind the project is non-fatal, so we intentionally ignore it.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 try:
 data = bytes(data)
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

pass
# Best-effort conversion to bytes; if this fails, fall back to using the
# original data and let the subsequent fromWkb call handle it.
logger.debug(
"Failed to convert WKB data to bytes in _geometry_from_value",
exc_info=True,
 )
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+657
 to 
+671

 except Exception:
 pass
 try:
 epsg = crs_info.to_epsg()
 if epsg:
 return QgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
 except Exception:
 pass
 if isinstance(crs_info, str):
 try:
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptException:
pass
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptException:
pass
exceptExceptionase:
logger.debug("Failed to create CRS from WKT using createFromWkt: %r (%s)", text, e)
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptExceptionase:
logger.debug("Failed to derive EPSG from CRS info %r: %s", crs_info, e)
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptExceptionase:
logger.debug("Failed to construct CRS from string %r: %s", crs_info, e)
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+657
 to 
+671

 except Exception:
 pass
 try:
 epsg = crs_info.to_epsg()
 if epsg:
 return QgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
 except Exception:
 pass
 if isinstance(crs_info, str):
 try:
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptException:
pass
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptException:
pass
exceptExceptionasexc:
logger.debug(
"Failed to create CRS from WKT using legacy createFromWkt: %s",
exc,
 )
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptExceptionasexc:
logger.debug(
"Failed to obtain EPSG code from CRS info %r: %s",
crs_info,
exc,
 )
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptExceptionasexc:
logger.debug(
"Failed to construct CRS from string %r: %s",
crs_info,
exc,
 )
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 elif isinstance(val, np.generic):
 try:
 val = val.item()
 except Exception:
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# If conversion to a native Python scalar fails, keep the original numpy scalar.
Copilot uses AI. Check for mistakes.

@lachlangrose
lachlangrose
merged commit ea99e14
into
dev-0.1.12Jan 30, 2026

1 check passed

@lachlangrose
lachlangrose
deleted the

data_converter_tool

branch
January 30, 2026 04:15
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@lachlangroselachlangroselachlangrose approved these changes

Assignees


No one assigned


Labels

qualityUI

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@rabii-chaarani@lachlangrose

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Data converter Widget by rabii-chaarani · Pull Request #76 · Loop3D/plugin_loopstructural · GitHub
Skip to content
Data converter Widget - #76
Merged
lachlangrose merged 20 commits into
dev-0.1.12from 
data_converter_tool
Jan 30, 2026
Merged
Data converter Widget#76
lachlangrose merged 20 commits into
dev-0.1.12from 
data_converter_tool
Conversation
@rabii-chaarani
@rabii-chaaranirabii-chaarani
commented
Jan 27, 2026

Copy link


Copy Markdown

Member
 No description provided. 
CopilotAI
reviewed
Jan 27, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
This pull request introduces a comprehensive "Data Converter Widget" that enables users to convert geological datasets for use within LoopStructural through both automatic and manual conversion workflows.
Changes:
  • Added a new data conversion widget with automatic and manual conversion modes
  • Implemented utility functions to convert GeoPandas GeoDataFrames and pandas DataFrames to QGIS vector layers
  • Integrated the data conversion widget into the main Loop widget interface
  • Added tests for the NTGS configuration model
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 17 comments.
Show a summary per file
FileDescription
tests/unit/gui/test_ntgs_configuration_model.pyUnit tests for configuration model (imports incorrect class names)
loopstructural/main/vectorLayerWrapper.pyNew functions to convert DataFrames/GeoDataFrames to QGIS layers
loopstructural/gui/loop_widget.pyIntegration of data conversion widget into main interface
loopstructural/gui/data_conversion/data_conversion_widget.pyMain widget implementation with automatic and manual conversion modes
loopstructural/gui/data_conversion/configuration.pyConfiguration state management for NTGS data types
loopstructural/gui/data_conversion/init.pyPackage initialization exposing DataConversionWidget
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadtests/unit/gui/test_ntgs_configuration_model.py
@@ -0,0 +1,33 @@
import pytest

from loopstructural.gui.data_conversion.configuration import NtgsConfig, NtgsConfigurationModel
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test imports NtgsConfig and NtgsConfigurationModel, but the actual configuration module defines these classes as Config and ConfigurationState. This import will fail at runtime. The imports should be updated to match the actual class names in loopstructural/gui/data_conversion/configuration.py.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+750
 to 
+764

 for _, row in df.iterrows():
 feat = QgsFeature(layer.fields())
 attrs = []
 for column in df.columns:
 val = row[column]
 if pd.isna(val):
 val = None
 elif isinstance(val, np.generic):
 try:
 val = val.item()
 except Exception:
 pass
 attrs.append(val)
 feat.setAttributes(attrs)
 features.append(feat)
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using iterrows() is generally inefficient for large DataFrames as it creates a new Series object for each row. For better performance with large datasets, consider using itertuples() or vectorized operations where possible.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/loop_widget.py
Outdated

 self.data_conversion_widget = DataConversionWidget(
 self,
 data_manager=self.data_manager,
 project=self.data_manager.project if self.data_manager else None,
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accessing self.data_manager.project could raise an AttributeError if data_manager is not None but lacks a project attribute. Consider using getattr(self.data_manager, 'project', None) for safer attribute access.

Suggested change

project=self.data_manager.projectifself.data_managerelseNone,
project=getattr(self.data_manager, "project", None)ifself.data_managerelseNone,
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/data_conversion/configuration.py
Outdated

Comment threadloopstructural/main/vectorLayerWrapper.py
 return layer


def QgsLayerFromDataFrame(dataframe, layer_name: str = "Converted Table"):
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function name QgsLayerFromDataFrame uses PascalCase, which is inconsistent with existing functions in this file like qgsLayerToGeoDataFrame, qgsLayerToDataFrame, and qgsRasterToGdalDataset that use camelCase. Consider renaming to qgsLayerFromDataFrame to maintain consistency with the existing naming convention in this file.

Suggested change

defQgsLayerFromDataFrame(dataframe, layer_name: str="Converted Table"):
defqgsLayerFromDataFrame(dataframe, layer_name: str="Converted Table"):
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 try:
 if temp.createFromWkt(text):
 return temp
 except Exception:
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
Comment threadloopstructural/main/vectorLayerWrapper.py
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# Ignore failures when interpreting CRS from string; fall back to default CRS.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+713
 to 
+714

 except Exception:
 pass
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
exceptExceptionasexc:
logger.debug("Failed to convert numpy scalar %r using .item(): %s", val, exc)
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+760
 to 
+761

 except Exception:
 pass
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
exceptExceptionasexc:
# Fall back to the original NumPy scalar if conversion fails.
logger.debug("Failed to convert NumPy scalar %r with .item(): %s", val, exc)
Copilot uses AI. Check for mistakes.

@lachlangrose
lachlangrose
commented
Jan 28, 2026

Copy link


Copy Markdown

Member
Can you make the data converter its own dialog or dock widget? I think it is an addition to the modelling process rather than a first step as you can use the modelling without using the data conversion. Also do you have a demo dataset I can use to test it?
@lachlangrose
lachlangrose
commented
Jan 28, 2026•

edited by rabii-chaarani

Loading

Copy link


Copy Markdown

Member
@rabii-chaarani looks good, couple of easy things
  •  use the column matcher from main.helper to automatically guess the appropriate file from the available files.
  •  Restrict the layer types to the appropriate geometries e.g. geology should be polygon not line or point, fault/fold should be line and structure points.
  •  Don't add xyz to the fault layer as they are a line layer, maybe add it as centre x, centre y, centre z.
Future idea
  •  Add a logging capabilities to report to the user what was done by the tool. E.g. column x changed to column y, create a file from a layer etc.
  •  Use the QGIS layers directly, not the underlying shapefile as the underlying shapefile is not compatible with filters.
CopilotAI
reviewed
Jan 30, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated 13 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadloopstructural/main/m2l_api.py

Comment on lines

540
 to 
541

 if unit_name_field != 'UNITNAME' and unit_name_field in geology_gdf.columns:
 geology_gdf = geology_gdf.rename(columns={unit_name_field: 'UNITNAME'})
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The unit name field is being renamed twice: once at line 492-494 and again at lines 540-541. This is redundant and could lead to confusion. The second rename should be removed as the first one already handles this transformation.

Suggested change

ifunit_name_field!='UNITNAME'andunit_name_fieldingeology_gdf.columns:
geology_gdf=geology_gdf.rename(columns={unit_name_field: 'UNITNAME'})
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/m2l_api.py
Outdated

 )
 sampled_contacts_gdf = qgsLayerToGeoDataFrame(sampled_contacts)
 structure_gdf = qgsLayerToGeoDataFrame(structure)
 structure_gdf = qgsLayerToGeoDataFrame(structure) 
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing whitespace found on line 460. This should be removed to maintain code cleanliness and consistency with the codebase style.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/data_conversion/data_conversion_widget.py
Outdated

Comment threadloopstructural/gui/map2loop_tools/thickness_calculator_widget.py
Comment threadloopstructural/gui/data_conversion/configuration.py
Outdated

Comment threadloopstructural/gui/data_conversion/data_conversion_widget.py
 if self.project is not None:
 try:
 combo.setProject(self.project)
 except Exception:
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# Some QGIS/Qt environments may not support setProject or may raise here;
# failure to bind the project is non-fatal, so we intentionally ignore it.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 try:
 data = bytes(data)
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

pass
# Best-effort conversion to bytes; if this fails, fall back to using the
# original data and let the subsequent fromWkb call handle it.
logger.debug(
"Failed to convert WKB data to bytes in _geometry_from_value",
exc_info=True,
 )
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+657
 to 
+671

 except Exception:
 pass
 try:
 epsg = crs_info.to_epsg()
 if epsg:
 return QgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
 except Exception:
 pass
 if isinstance(crs_info, str):
 try:
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptException:
pass
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptException:
pass
exceptExceptionase:
logger.debug("Failed to create CRS from WKT using createFromWkt: %r (%s)", text, e)
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptExceptionase:
logger.debug("Failed to derive EPSG from CRS info %r: %s", crs_info, e)
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptExceptionase:
logger.debug("Failed to construct CRS from string %r: %s", crs_info, e)
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+657
 to 
+671

 except Exception:
 pass
 try:
 epsg = crs_info.to_epsg()
 if epsg:
 return QgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
 except Exception:
 pass
 if isinstance(crs_info, str):
 try:
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptException:
pass
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptException:
pass
exceptExceptionasexc:
logger.debug(
"Failed to create CRS from WKT using legacy createFromWkt: %s",
exc,
 )
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptExceptionasexc:
logger.debug(
"Failed to obtain EPSG code from CRS info %r: %s",
crs_info,
exc,
 )
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptExceptionasexc:
logger.debug(
"Failed to construct CRS from string %r: %s",
crs_info,
exc,
 )
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 elif isinstance(val, np.generic):
 try:
 val = val.item()
 except Exception:
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# If conversion to a native Python scalar fails, keep the original numpy scalar.
Copilot uses AI. Check for mistakes.

@lachlangrose
lachlangrose
merged commit ea99e14
into
dev-0.1.12Jan 30, 2026

1 check passed

@lachlangrose
lachlangrose
deleted the

data_converter_tool

branch
January 30, 2026 04:15
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@lachlangroselachlangroselachlangrose approved these changes

Assignees


No one assigned


Labels

qualityUI

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@rabii-chaarani@lachlangrose

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
Data converter Widget by rabii-chaarani · Pull Request #76 · Loop3D/plugin_loopstructural · GitHub
Skip to content
Data converter Widget - #76
Merged
lachlangrose merged 20 commits into
dev-0.1.12from 
data_converter_tool
Jan 30, 2026
Merged
Data converter Widget#76
lachlangrose merged 20 commits into
dev-0.1.12from 
data_converter_tool
Conversation
@rabii-chaarani
@rabii-chaaranirabii-chaarani
commented
Jan 27, 2026

Copy link


Copy Markdown

Member
 No description provided. 
CopilotAI
reviewed
Jan 27, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
This pull request introduces a comprehensive "Data Converter Widget" that enables users to convert geological datasets for use within LoopStructural through both automatic and manual conversion workflows.
Changes:
  • Added a new data conversion widget with automatic and manual conversion modes
  • Implemented utility functions to convert GeoPandas GeoDataFrames and pandas DataFrames to QGIS vector layers
  • Integrated the data conversion widget into the main Loop widget interface
  • Added tests for the NTGS configuration model
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 17 comments.
Show a summary per file
FileDescription
tests/unit/gui/test_ntgs_configuration_model.pyUnit tests for configuration model (imports incorrect class names)
loopstructural/main/vectorLayerWrapper.pyNew functions to convert DataFrames/GeoDataFrames to QGIS layers
loopstructural/gui/loop_widget.pyIntegration of data conversion widget into main interface
loopstructural/gui/data_conversion/data_conversion_widget.pyMain widget implementation with automatic and manual conversion modes
loopstructural/gui/data_conversion/configuration.pyConfiguration state management for NTGS data types
loopstructural/gui/data_conversion/init.pyPackage initialization exposing DataConversionWidget
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadtests/unit/gui/test_ntgs_configuration_model.py
@@ -0,0 +1,33 @@
import pytest

from loopstructural.gui.data_conversion.configuration import NtgsConfig, NtgsConfigurationModel
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test imports NtgsConfig and NtgsConfigurationModel, but the actual configuration module defines these classes as Config and ConfigurationState. This import will fail at runtime. The imports should be updated to match the actual class names in loopstructural/gui/data_conversion/configuration.py.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+750
 to 
+764

 for _, row in df.iterrows():
 feat = QgsFeature(layer.fields())
 attrs = []
 for column in df.columns:
 val = row[column]
 if pd.isna(val):
 val = None
 elif isinstance(val, np.generic):
 try:
 val = val.item()
 except Exception:
 pass
 attrs.append(val)
 feat.setAttributes(attrs)
 features.append(feat)
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using iterrows() is generally inefficient for large DataFrames as it creates a new Series object for each row. For better performance with large datasets, consider using itertuples() or vectorized operations where possible.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/loop_widget.py
Outdated

 self.data_conversion_widget = DataConversionWidget(
 self,
 data_manager=self.data_manager,
 project=self.data_manager.project if self.data_manager else None,
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accessing self.data_manager.project could raise an AttributeError if data_manager is not None but lacks a project attribute. Consider using getattr(self.data_manager, 'project', None) for safer attribute access.

Suggested change

project=self.data_manager.projectifself.data_managerelseNone,
project=getattr(self.data_manager, "project", None)ifself.data_managerelseNone,
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/data_conversion/configuration.py
Outdated

Comment threadloopstructural/main/vectorLayerWrapper.py
 return layer


def QgsLayerFromDataFrame(dataframe, layer_name: str = "Converted Table"):
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function name QgsLayerFromDataFrame uses PascalCase, which is inconsistent with existing functions in this file like qgsLayerToGeoDataFrame, qgsLayerToDataFrame, and qgsRasterToGdalDataset that use camelCase. Consider renaming to qgsLayerFromDataFrame to maintain consistency with the existing naming convention in this file.

Suggested change

defQgsLayerFromDataFrame(dataframe, layer_name: str="Converted Table"):
defqgsLayerFromDataFrame(dataframe, layer_name: str="Converted Table"):
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 try:
 if temp.createFromWkt(text):
 return temp
 except Exception:
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
Comment threadloopstructural/main/vectorLayerWrapper.py
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# Ignore failures when interpreting CRS from string; fall back to default CRS.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+713
 to 
+714

 except Exception:
 pass
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
exceptExceptionasexc:
logger.debug("Failed to convert numpy scalar %r using .item(): %s", val, exc)
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+760
 to 
+761

 except Exception:
 pass
CopilotAIJan 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
exceptExceptionasexc:
# Fall back to the original NumPy scalar if conversion fails.
logger.debug("Failed to convert NumPy scalar %r with .item(): %s", val, exc)
Copilot uses AI. Check for mistakes.

@lachlangrose
lachlangrose
commented
Jan 28, 2026

Copy link


Copy Markdown

Member
Can you make the data converter its own dialog or dock widget? I think it is an addition to the modelling process rather than a first step as you can use the modelling without using the data conversion. Also do you have a demo dataset I can use to test it?
@lachlangrose
lachlangrose
commented
Jan 28, 2026•

edited by rabii-chaarani

Loading

Copy link


Copy Markdown

Member
@rabii-chaarani looks good, couple of easy things
  •  use the column matcher from main.helper to automatically guess the appropriate file from the available files.
  •  Restrict the layer types to the appropriate geometries e.g. geology should be polygon not line or point, fault/fold should be line and structure points.
  •  Don't add xyz to the fault layer as they are a line layer, maybe add it as centre x, centre y, centre z.
Future idea
  •  Add a logging capabilities to report to the user what was done by the tool. E.g. column x changed to column y, create a file from a layer etc.
  •  Use the QGIS layers directly, not the underlying shapefile as the underlying shapefile is not compatible with filters.
CopilotAI
reviewed
Jan 30, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated 13 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadloopstructural/main/m2l_api.py

Comment on lines

540
 to 
541

 if unit_name_field != 'UNITNAME' and unit_name_field in geology_gdf.columns:
 geology_gdf = geology_gdf.rename(columns={unit_name_field: 'UNITNAME'})
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The unit name field is being renamed twice: once at line 492-494 and again at lines 540-541. This is redundant and could lead to confusion. The second rename should be removed as the first one already handles this transformation.

Suggested change

ifunit_name_field!='UNITNAME'andunit_name_fieldingeology_gdf.columns:
geology_gdf=geology_gdf.rename(columns={unit_name_field: 'UNITNAME'})
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/m2l_api.py
Outdated

 )
 sampled_contacts_gdf = qgsLayerToGeoDataFrame(sampled_contacts)
 structure_gdf = qgsLayerToGeoDataFrame(structure)
 structure_gdf = qgsLayerToGeoDataFrame(structure) 
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing whitespace found on line 460. This should be removed to maintain code cleanliness and consistency with the codebase style.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/gui/data_conversion/data_conversion_widget.py
Outdated

Comment threadloopstructural/gui/map2loop_tools/thickness_calculator_widget.py
Comment threadloopstructural/gui/data_conversion/configuration.py
Outdated

Comment threadloopstructural/gui/data_conversion/data_conversion_widget.py
 if self.project is not None:
 try:
 combo.setProject(self.project)
 except Exception:
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# Some QGIS/Qt environments may not support setProject or may raise here;
# failure to bind the project is non-fatal, so we intentionally ignore it.
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 try:
 data = bytes(data)
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

pass
# Best-effort conversion to bytes; if this fails, fall back to using the
# original data and let the subsequent fromWkb call handle it.
logger.debug(
"Failed to convert WKB data to bytes in _geometry_from_value",
exc_info=True,
 )
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+657
 to 
+671

 except Exception:
 pass
 try:
 epsg = crs_info.to_epsg()
 if epsg:
 return QgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
 except Exception:
 pass
 if isinstance(crs_info, str):
 try:
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptException:
pass
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptException:
pass
exceptExceptionase:
logger.debug("Failed to create CRS from WKT using createFromWkt: %r (%s)", text, e)
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptExceptionase:
logger.debug("Failed to derive EPSG from CRS info %r: %s", crs_info, e)
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptExceptionase:
logger.debug("Failed to construct CRS from string %r: %s", crs_info, e)
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py

Comment on lines

+657
 to 
+671

 except Exception:
 pass
 try:
 epsg = crs_info.to_epsg()
 if epsg:
 return QgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
 except Exception:
 pass
 if isinstance(crs_info, str):
 try:
 temp = QgsCoordinateReferenceSystem(crs_info)
 if temp.isValid():
 return temp
 except Exception:
 pass
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
pass
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptException:
pass
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptException:
pass
exceptExceptionasexc:
logger.debug(
"Failed to create CRS from WKT using legacy createFromWkt: %s",
exc,
 )
try:
epsg=crs_info.to_epsg()
ifepsg:
returnQgsCoordinateReferenceSystem.fromEpsgId(int(epsg))
exceptExceptionasexc:
logger.debug(
"Failed to obtain EPSG code from CRS info %r: %s",
crs_info,
exc,
 )
ifisinstance(crs_info, str):
try:
temp=QgsCoordinateReferenceSystem(crs_info)
iftemp.isValid():
returntemp
exceptExceptionasexc:
logger.debug(
"Failed to construct CRS from string %r: %s",
crs_info,
exc,
 )
Copilot uses AI. Check for mistakes.

Comment threadloopstructural/main/vectorLayerWrapper.py
 elif isinstance(val, np.generic):
 try:
 val = val.item()
 except Exception:
CopilotAIJan 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

exceptException:
exceptException:
# If conversion to a native Python scalar fails, keep the original numpy scalar.
Copilot uses AI. Check for mistakes.

@lachlangrose
lachlangrose
merged commit ea99e14
into
dev-0.1.12Jan 30, 2026

1 check passed

@lachlangrose
lachlangrose
deleted the

data_converter_tool

branch
January 30, 2026 04:15
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@lachlangroselachlangroselachlangrose approved these changes

Assignees


No one assigned


Labels

qualityUI

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@rabii-chaarani@lachlangrose