Lightweight, dependency-free buffer analysis engine for detecting MIME types (magic bytes) and quick heuristic suspicious-patterns in buffers.
- Fast magic-bytes based MIME type detection
- Heuristic suspicious pattern scanning (HTML/JS/SQL/shell snippets)
- Configurable analysis depth and file-size skipping
- Minimal, dependency-free TypeScript implementation
npm install buffer-analysis-engine --save
# or
pnpm add buffer-analysis-engineimport{BufferAnalysisEngine,analyzeBuffer,getBufferAnalysisEngine,resetBufferAnalysisEngine,MAGIC_BYTES_SIGNATURES,analyzeSuspiciousPatterns,getPatternsByCategory,}from'buffer-analysis-engine';constengine=newBufferAnalysisEngine();constbuf=Buffer.from([0xff,0xd8,0xff,0xe0]);// instance APIconstresult=engine.analyzeBuffer(buf,'photo.jpg');console.log(result.detectedMimeType);// "image/jpeg"// convenience: per-call config uses a temporary engine and thus respects the supplied optionsconstoneOff=analyzeBuffer(buf,'photo.jpg',{enableMagicBytesDetection: true});// direct suspicious pattern analysisconstmaliciousBuf=Buffer.from('<script>alert(1)</script>');constpatternResult=analyzeSuspiciousPatterns(maliciousBuf);console.log(patternResult.hasSuspicious);// trueconsole.log(patternResult.patterns);// ['HTML Script Tag', 'HTML Script Close Tag']// get patterns by categoryconstsqlPatterns=getPatternsByCategory('SQL');console.log(sqlPatterns.length);// number of SQL-related patterns// reset the global instance (useful in tests or to reconfigure)resetBufferAnalysisEngine();// inspect available signaturesconsole.log(Object.keys(MAGIC_BYTES_SIGNATURES));Note:
getBufferAnalysisEngine()returns a global singleton. If you pass aconfigtogetBufferAnalysisEngine()it only affects the first initialization. To recreate the global instance, callresetBufferAnalysisEngine().
BufferAnalysisEngine
- constructor(config?: BufferAnalysisConfig, logger?: SimpleLogger)
- analyzeBuffer(buffer: Buffer, filename?: string): BufferAnalysisResult
- getConfig(), updateConfig(), enable(), disable(), isEnabled()
analyzeBuffer(buffer, filename?, config?) — convenience function. If
configis provided it creates a temporary engine for that call (won't mutate global singleton).getBufferAnalysisEngine(config?, logger?) — returns a global singleton instance (first call initializes it with given config).
resetBufferAnalysisEngine() — reset the global singleton (useful in tests or when reconfiguring at runtime).
MAGIC_BYTES_SIGNATURES — exported map of mime types -> magic byte signatures
addMagicBytesSignature(mimeType, signature) — add a magic signature at runtime
removeMagicBytesSignatures(mimeType) — remove all signatures for a mime type
SUSPICIOUS_PATTERNS — exported array of suspicious pattern definitions
analyzeSuspiciousPatterns(buffer, maxAnalysisDepth?) — analyze buffer for security threats
getAllSuspiciousPatternNames() — get all pattern names
getSuspiciousPatternCount() — get total number of patterns
getPatternsByCategory(category) — filter patterns by category (e.g., 'HTML', 'SQL')
analyzeStream(readable, filename?, config?) — analyze a Readable or async iterable stream (returns a Promise)
expressBufferAnalysisMiddleware(opts) / koaBufferAnalysisMiddleware(opts) — middleware factories for Express and Koa to attach analysis results to requests
The engine includes a comprehensive set of patterns to detect various security threats including XSS, SQL injection, command injection, and other malicious content.
import{analyzeSuspiciousPatterns,getPatternsByCategory}from'buffer-analysis-engine';// Analyze a buffer for suspicious patterns with contextconstbuffer=Buffer.from('<script>alert("XSS")</script> DROP TABLE users');constmimeType='text/html';// Optional: filters inapplicable patternsconstmaxDepth=1024;constthreshold=1;constresult=analyzeSuspiciousPatterns(buffer,mimeType,maxDepth,threshold);console.log(result.hasSuspicious);// trueconsole.log(result.patterns);// ['HTML Script Tag', 'SQL Drop Command', ...]console.log(result.score);// Total risk score (e.g., 5)// Get patterns by categoryconsthtmlPatterns=getPatternsByCategory('HTML');constsqlPatterns=getPatternsByCategory('SQL');constjsPatterns=getPatternsByCategory('JavaScript');// Get all pattern namesimport{getAllSuspiciousPatternNames}from'buffer-analysis-engine';constallNames=getAllSuspiciousPatternNames();console.log(`Total patterns: ${allNames.length}`);The suspicious patterns are organized into categories:
- HTML/XSS: Script tags, event handlers, iframe/object/embed tags
- JavaScript: Eval, alert, confirm, Function constructor, localStorage access
- SQL Injection: DROP/TRUNCATE/UNION/SELECT statements, comments, tautologies
- Command Injection: Exec, system, shell commands, command operators
- PHP: Include/require, eval, create_function
- Python: Exec, eval, subprocess, import calls
- File System: Path traversal, passwd/shadow files
- Encoding: Base64, URL encoding functions
// Only analyze first 1KB of a large buffer, with threshold 3constresult=analyzeSuspiciousPatterns(largeBuffer,'text/plain',1024,3);You can analyze Readable streams (or async iterables of Buffer chunks) without buffering the entire file into memory. The convenience function analyzeStream(readable, filename?, config?) returns a promise resolving to BufferAnalysisResult.
Example:
import{analyzeStream}from'buffer-analysis-engine';import{Readable}from'stream';constr=Readable.from([Buffer.from([0xff,0xd8,0xff,0xe0])]);constres=awaitanalyzeStream(r,'photo.jpg');Notes:
- For suspicious-pattern analysis, the implementation buffers up to
maxAnalysisDepthbytes (default 1 MiB). - For magic-bytes detection, the stream reader continues reading until the required buffer depth is met, ensuring accurate detection even for patterns deeper in the file (if configured).
We provide tiny middleware factories to help integrate the engine into Node.js web services.
Example (Express):
importexpressfrom'express';import{expressBufferAnalysisMiddleware}from'buffer-analysis-engine';constapp=express();// Use after body parsers so body is available as Buffer/stringapp.use(expressBufferAnalysisMiddleware({attachProperty: 'bufferAnalysis'}));app.post('/upload',(req,res)=>{// req.bufferAnalysis is attached by middlewareres.json({mime: req.bufferAnalysis?.detectedMimeType});});If you need middleware to inspect raw streams (before body parsing) you can enable consumeRequestStream: true but beware this consumes the request body; downstream handlers must read from req.rawBody if you attach it.
You can add and remove signatures at runtime:
import{addMagicBytesSignature,removeMagicBytesSignatures}from'buffer-analysis-engine';addMagicBytesSignature('application/x-custom',[0x01,0x02,0x03]);| Option | Type | Default | Description |
|---|---|---|---|
| enableMagicBytesDetection | boolean | true | Toggle magic-bytes/MIME detection |
| enableSuspiciousPatternAnalysis | boolean | true | Toggle suspicious pattern scanning |
| maxAnalysisDepth | number | 1 MiB | Max bytes to scan for patterns |
| skipLargeFiles | boolean | true | If true, files larger than maxFileSize are skipped |
| maxFileSize | number | 50 MiB | Threshold for skipping large files |
| suspiciousThreshold | number | 1 | Minimum score to flag as suspicious |
| mimeTypeSpecificConfig | object | {} | Per-MIME type configuration overrides |
The engine supports granular configuration for different file types and security requirements.
Suspicious Thresholds:
Patterns have assigned weights (e.g., alert() = 1, DROP TABLE = 3). Content is flagged only if the total score meets or exceeds suspiciousThreshold.
MIME-Type Overrides:
You can apply stricter or looser rules for specific file types using mimeTypeSpecificConfig:
constengine=newBufferAnalysisEngine({suspiciousThreshold: 1,// Default strictmimeTypeSpecificConfig: {'text/javascript': {suspiciousThreshold: 5,// Allow some patterns in JS filesmaxAnalysisDepth: 5*1024*1024,// Scan deeper in JS},},});- Run tests:
npm test - Build:
npm run build
We use Vitest for unit tests and TypeScript for static types.
Contributions are welcome — please open issues or PRs with clear descriptions and tests.
MIT