Edge-native PII redaction and tokenization library for protecting sensitive data in AI applications. Built on Web Standards with pluggable country-specific modules.
A lightweight AI security layer for detecting and protecting Personally Identifiable Information (PII) in code and data. Inspired by Japanese noren curtains that protect without closing the door, this library provides edge-native PII redaction and tokenization built on Web Standards (WHATWG Streams, WebCrypto, fetch).
This skill helps you work with the Noren library to detect, redact, and tokenize sensitive data in your applications. It provides guidance on:
When working with Noren, recognize its monorepo structure:
The library is built on Web Standards only (no Node.js-specific APIs beyond standard globals).
For fresh setup:
```sh
git clone [repository-url]
cd noren
pnpm i
pnpm build
pnpm test
pnpm check
```
For individual package work:
```sh
cd packages/[package-name]
pnpm test
pnpm build
```
**Basic PII Redaction:**
```javascript
import { detectPII, maskPII } from '@himorishige/noren-core'
import { registerJPDetectors } from '@himorishige/noren-plugin-jp'
// Register country-specific detectors
registerJPDetectors()
// Detect PII
const text = "Contact: [email protected], 090-1234-5678"
const results = detectPII(text)
// Mask sensitive data
const masked = maskPII(text, results)
```
**Stream Processing for Large Files:**
```javascript
import { createRedactionStream } from '@himorishige/noren-core'
import { createReadStream } from 'fs'
const inputStream = createReadStream('data.txt')
const redactionStream = createRedactionStream()
const outputStream = createWriteStream('redacted.txt')
await inputStream
.pipeThrough(new TextDecoderStream())
.pipeThrough(redactionStream)
.pipeThrough(new TextEncoderStream())
.pipeTo(outputStream)
```
**HMAC-based Tokenization:**
```javascript
import { tokenizePII } from '@himorishige/noren-core'
// Requires 32+ character HMAC key
const hmacKey = 'your-secure-32-character-key-here'
const text = "User email: [email protected]"
const tokenized = await tokenizePII(text, hmacKey, {
format: 'base64url' // More secure than legacy formats
})
// Result: "User email: [EMAIL:Abc123XYZ...]"
```
**Detection Sensitivity Levels:**
```javascript
import { detectPII, SENSITIVITY } from '@himorishige/noren-core'
// Strict mode: high-confidence only
const strictResults = detectPII(text, {
minConfidence: SENSITIVITY.STRICT // 0.8
})
// Balanced mode: moderate confidence
const balancedResults = detectPII(text, {
minConfidence: SENSITIVITY.BALANCED // 0.6
})
// Relaxed mode: catch potential false negatives
const relaxedResults = detectPII(text, {
minConfidence: SENSITIVITY.RELAXED // 0.4
})
```
**Environment-Aware Configuration:**
```javascript
import { AllowDenyManager } from '@himorishige/noren-core'
const manager = new AllowDenyManager()
// Add custom allowlist patterns
manager.addToAllowlist('email', '[email protected]')
manager.addToAllowlist('ip', '10.0.0.0/8') // Private network
// Default test patterns excluded automatically in dev/test environments:
// - example.com, localhost, [email protected]
// - Private IPs (10.x.x.x, 192.168.x.x, 127.0.0.1)
const results = detectPII(text, {
allowDenyManager: manager
})
```
**Protect API Keys, JWT Tokens, and Cookies:**
```javascript
import { registerSecurityDetectors } from '@himorishige/noren-plugin-security'
registerSecurityDetectors({
cookieAllowlist: ['session_token'] // Don't mask these cookies
})
const headers = `
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
X-API-Key: sk-1234567890abcdef
Cookie: session_token=abc123; tracking_id=xyz789
`
const results = detectPII(headers)
// Detects JWT, API key, and tracking cookie (but not session_token)
```
**Dynamic Policy Updates:**
```javascript
import { loadDictionary } from '@himorishige/noren-dict-reloader'
// Load from remote URL with ETag caching
const dict = await loadDictionary('https://api.company.com/pii-policy.json', {
ttl: 3600, // Cache for 1 hour
validateFn: (data) => data.version === '1.0' // Custom validation
})
// Register custom patterns
dict.patterns.forEach(pattern => {
registerCustomDetector(pattern)
})
// Reload automatically on updates
setInterval(async () => {
const updated = await loadDictionary(url, { ttl: 3600 })
if (updated !== dict) {
// Policy changed, re-register detectors
}
}, 300000) // Check every 5 minutes
```
**Using Development Tools:**
```javascript
import { benchmark, evaluateAccuracy } from '@himorishige/noren-devtools'
// Performance benchmarking
const results = await benchmark(() => {
return detectPII(largeText)
}, {
iterations: 1000,
warmup: 100
})
console.log(`Avg time: ${results.avgTime}ms`)
console.log(`Memory delta: ${results.memoryDelta}MB`)
// Accuracy evaluation
const groundTruth = [
{ text: "Call 090-1234-5678", expected: ['090-1234-5678'] },
{ text: "Email: [email protected]", expected: ['[email protected]'] }
]
const evaluation = evaluateAccuracy(groundTruth, detectPII)
console.log(`Precision: ${evaluation.precision}`)
console.log(`Recall: ${evaluation.recall}`)
console.log(`F1 Score: ${evaluation.f1}`)
```
**Example with Hono:**
```javascript
import { Hono } from 'hono'
import { maskPII, detectPII } from '@himorishige/noren-core'
const app = new Hono()
app.post('/api/sanitize', async (c) => {
const body = await c.req.json()
const results = detectPII(body.text)
const sanitized = maskPII(body.text, results)
return c.json({
original: body.text,
sanitized,
detectedTypes: [...new Set(results.map(r => r.type))]
})
})
export default app
```
**Running Tests:**
```sh
pnpm test
cd packages/noren-core
pnpm test
pnpm check
```
**Creating Releases:**
```sh
pnpm changeset
pnpm changeset:status
pnpm changeset:version
pnpm changeset:publish
```
**PR Canary Testing:**
```sh
npm install @himorishige/noren-core@canary
```
Run example scripts to understand common patterns:
```sh
node examples/basic-redact.mjs # Basic PII redaction
node examples/tokenize.mjs # Tokenization
node examples/stream-redact.mjs # Streaming
node examples/security-demo.mjs # Security plugin
node examples/dictionary-demo.mjs # Custom dictionaries
node examples/hono-server.mjs # Web server integration
node examples/benchmark-demo.mjs # Performance benchmarking
node examples/evaluation-demo.mjs # Accuracy evaluation
```
Leave a review
No reviews yet. Be the first to review this skill!
# Download SKILL.md from killerskills.ai/api/skills/noren-ai-security-layer/raw