Step-by-step tutorial for creating reusable custom field sections in Glazed
Custom field sections address the common challenge of duplicating field definitions across multiple CLI commands. Instead of copying the same flags for logging, database connections, or API configurations across commands, field sections provide reusable components that encapsulate related fields and their validation logic.
This tutorial demonstrates building a production-ready logging section that can be reused across any Glazed command, providing consistent configuration interfaces and behavior throughout an application.
This tutorial covers:
Comprehensive logging configuration for production applications requires multiple field categories:
Basic Features:
Production Features:
Field sections eliminate the need to duplicate these 7+ flags across every command by defining the configuration once and reusing it throughout the application.
Production logging sections require fields that address both developer and operational requirements:
Core Fields:
Production Fields:
mkdir glazed-logging-section
cd glazed-logging-section
go mod init glazed-logging-section
go get github.com/go-go-golems/glazed
go get github.com/spf13/cobra
go get github.com/rs/zerolog
The project structure separates field definitions from business logic for maintainability:
glazed-logging-section/
├── main.go # Demo commands showing section usage
├── logging/
│ ├── section.go # Field definitions and section creation
│ ├── settings.go # Type-safe configuration struct
│ └── init.go # Logger setup and initialization
└── go.mod
This separation enables independent testing of field validation and provides clear initialization patterns for applications using the section.
Create logging/settings.go:
The settings struct defines the section's configuration interface, using struct tags to map CLI fields to Go fields. This struct serves as both the field binding target and the configuration container for logger initialization.
package logging
import (
"fmt"
"io"
"os"
"strings"
"time"
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
)
// LoggingSettings represents all logging configuration options.
// This struct serves as both the field binding target and the
// configuration container for logger initialization.
type LoggingSettings struct {
// Core logging settings - the 80% use case
Level string `glazed:"log-level"`
Format string `glazed:"log-format"`
File string `glazed:"log-file"`
// Developer convenience settings
WithCaller bool `glazed:"with-caller"`
Verbose bool `glazed:"verbose"`
}
// Validate checks if the logging settings are valid.
// Input validation prevents runtime failures from invalid configuration.
func (s *LoggingSettings) Validate() error {
// Validate log level - catch typos early
validLevels := []string{"debug", "info", "warn", "error", "fatal", "panic"}
if !contains(validLevels, s.Level) {
return fmt.Errorf("invalid log level '%s', must be one of: %s",
s.Level, strings.Join(validLevels, ", "))
}
// Validate log format - prevent silent failures in log parsing
validFormats := []string{"text", "json"}
if !contains(validFormats, s.Format) {
return fmt.Errorf("invalid log format '%s', must be one of: %s",
s.Format, strings.Join(validFormats, ", "))
}
return nil
}
// GetLogLevel converts string level to zerolog.Level.
// The verbose flag overrides the configured level for debugging convenience.
func (s *LoggingSettings) GetLogLevel() zerolog.Level {
// Verbose flag takes precedence over configured level
if s.Verbose {
return zerolog.DebugLevel
}
switch strings.ToLower(s.Level) {
case "debug":
return zerolog.DebugLevel
case "info":
return zerolog.InfoLevel
case "warn":
return zerolog.WarnLevel
case "error":
return zerolog.ErrorLevel
case "fatal":
return zerolog.FatalLevel
case "panic":
return zerolog.PanicLevel
default:
// Default to info level for balanced logging
return zerolog.InfoLevel
}
}
// GetWriter returns the appropriate writer for log output.
// Defaults to stderr to separate log output from program output.
func (s *LoggingSettings) GetWriter() (io.Writer, error) {
if s.File == "" {
// Use stderr for log output to avoid mixing with program output
return os.Stderr, nil
}
// Append to log files to preserve history across restarts
file, err := os.OpenFile(s.File, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0666)
if err != nil {
return nil, fmt.Errorf("failed to open log file '%s': %w", s.File, err)
}
return file, nil
}
// SetupLogger configures the global logger with these settings
func (s *LoggingSettings) SetupLogger() error {
// Validate settings first
if err := s.Validate(); err != nil {
return err
}
// Set log level
zerolog.SetGlobalLevel(s.GetLogLevel())
// Get writer
writer, err := s.GetWriter()
if err != nil {
return err
}
// Configure output format
var output io.Writer = writer
if s.Format == "text" {
// Pretty console output for text format
if s.File == "" { // Only if writing to stderr
output = zerolog.ConsoleWriter{
Out: writer,
TimeFormat: time.RFC3339,
NoColor: false,
}
}
}
// Create logger
logger := zerolog.New(output).With().Timestamp()
// Add caller information if requested
if s.WithCaller {
logger = logger.Caller()
}
// Set as global logger
log.Logger = logger.Logger()
return nil
}
// Helper function
func contains(slice []string, item string) bool {
for _, s := range slice {
if s == item {
return true
}
}
return false
}
Create logging/section.go:
The section definition specifies the CLI fields and their configuration options. Each field includes type, default values, validation rules, and help text.
package logging
import (
"github.com/go-go-golems/glazed/pkg/cmds/schema"
"github.com/go-go-golems/glazed/pkg/cmds/fields"
)
const (
// LoggingSlug is the unique identifier for this section.
LoggingSlug = "logging"
)
// NewLoggingSection creates a new field section for logging configuration.
func NewLoggingSection() (schema.Section, error) {
return schema.NewSection(
LoggingSlug,
"Logging Configuration",
schema.WithFields(
// Core logging fields - the ones everyone needs
fields.New(
"log-level",
fields.TypeChoice,
fields.WithHelp("Set the logging level"),
fields.WithDefault("info"), // Safe default - not too noisy, not too quiet
fields.WithChoices("debug", "info", "warn", "error", "fatal", "panic"),
fields.WithShortFlag("L"), // Capital L to avoid conflicts with -l (list)
),
fields.New(
"log-format",
fields.TypeChoice,
fields.WithHelp("Set the log output format"),
fields.WithDefault("text"), // Human-readable by default
fields.WithChoices("text", "json"), // JSON for log aggregation systems
),
fields.New(
"log-file",
fields.TypeString,
fields.WithHelp("Log file path (default: stderr)"),
fields.WithDefault(""), // Empty means stderr - explicit in help text
),
// Developer convenience fields
fields.New(
"with-caller",
fields.TypeBool,
fields.WithHelp("Include caller information in log entries"),
fields.WithDefault(false), // Off by default - performance impact
),
fields.New(
"verbose",
fields.TypeBool,
fields.WithHelp("Enable verbose logging (sets level to debug)"),
fields.WithDefault(false),
fields.WithShortFlag("v"), // Classic Unix convention
),
),
)
}
// NewLoggingSectionWithOptions creates a logging section with customization options
func NewLoggingSectionWithOptions(opts ...LoggingSectionOption) (schema.Section, error) {
config := &loggingSectionConfig{
defaultLevel: "info",
defaultFormat: "text",
}
// Apply options
for _, opt := range opts {
opt(config)
}
section, err := NewLoggingSection()
if err != nil {
return nil, err
}
// Modify defaults based on config
params := section.GetDefinitions()
if levelParam := params.Get("log-level"); levelParam != nil {
defaultLevel := interface{}(config.defaultLevel)
levelParam.Default = &defaultLevel
}
if formatParam := params.Get("log-format"); formatParam != nil {
defaultFormat := interface{}(config.defaultFormat)
formatParam.Default = &defaultFormat
}
// NOTE: RemoveFlag method doesn't exist in the current API.
// To implement conditional fields, you would need to create separate sections
// or build the section conditionally rather than removing fields after creation.
// For production code, use the basic NewLoggingSection() without RemoveFlag calls.
return section, nil
}
// Configuration options for the logging section
type loggingSectionConfig struct {
defaultLevel string
defaultFormat string
}
type LoggingSectionOption func(*loggingSectionConfig)
// WithDefaultLevel sets the default log level
func WithDefaultLevel(level string) LoggingSectionOption {
return func(c *loggingSectionConfig) {
c.defaultLevel = level
}
}
// WithDefaultFormat sets the default log format
func WithDefaultFormat(format string) LoggingSectionOption {
return func(c *loggingSectionConfig) {
c.defaultFormat = format
}
}
Create logging/init.go:
package logging
import (
"fmt"
"github.com/go-go-golems/glazed/pkg/cmds/schema"
"github.com/rs/zerolog/log"
)
// GetLoggingSettings extracts logging settings from parsed sections
func GetLoggingSettings(parsedSections *values.Values) (*LoggingSettings, error) {
settings := &LoggingSettings{}
if err := parsedSections.DecodeSectionInto(LoggingSlug, settings); err != nil {
return nil, fmt.Errorf("failed to initialize logging settings: %w", err)
}
return settings, nil
}
// InitializeLogging sets up logging from parsed sections
func InitializeLogging(parsedSections *values.Values) error {
settings, err := GetLoggingSettings(parsedSections)
if err != nil {
return err
}
if err := settings.SetupLogger(); err != nil {
return fmt.Errorf("failed to setup logger: %w", err)
}
log.Debug().
Str("level", settings.Level).
Str("format", settings.Format).
Str("file", settings.File).
Bool("with_caller", settings.WithCaller).
Bool("verbose", settings.Verbose).
Msg("Logging initialized")
return nil
}
// MustInitializeLogging sets up logging or panics on error
func MustInitializeLogging(parsedSections *values.Values) {
if err := InitializeLogging(parsedSections); err != nil {
panic(fmt.Sprintf("Failed to initialize logging: %v", err))
}
}
// SetupDefaultLogging configures logging with default settings (useful for testing)
func SetupDefaultLogging() error {
settings := &LoggingSettings{
Level: "info",
Format: "text",
File: "",
WithCaller: false,
Verbose: false,
}
return settings.SetupLogger()
}
Create main.go:
package main
import (
"context"
"fmt"
"os"
"time"
"glazed-logging-section/logging"
"github.com/go-go-golems/glazed/pkg/cli"
"github.com/go-go-golems/glazed/pkg/cmds"
"github.com/go-go-golems/glazed/pkg/cmds/schema"
"github.com/go-go-golems/glazed/pkg/cmds/fields"
"github.com/go-go-golems/glazed/pkg/middlewares"
"github.com/go-go-golems/glazed/pkg/settings"
"github.com/go-go-golems/glazed/pkg/types"
"github.com/rs/zerolog/log"
"github.com/spf13/cobra"
)
// ProcessDataCommand demonstrates using the logging section
type ProcessDataCommand struct {
*cmds.CommandDescription
}
type ProcessDataSettings struct {
InputFile string `glazed:"input-file"`
OutputPath string `glazed:"output-path"`
Workers int `glazed:"workers"`
DryRun bool `glazed:"dry-run"`
}
func (c *ProcessDataCommand) RunIntoGlazeProcessor(
ctx context.Context,
parsedSections *values.Values,
gp middlewares.Processor,
) error {
// Initialize logging first
if err := logging.InitializeLogging(parsedSections); err != nil {
return fmt.Errorf("failed to initialize logging: %w", err)
}
log.Info().Msg("Starting data processing command")
// Get command settings
settings := &ProcessDataSettings{}
if err := parsedSections.DecodeSectionInto(schema.DefaultSlug, settings); err != nil {
return err
}
log.Debug().
Str("input_file", settings.InputFile).
Str("output_file", settings.OutputFile).
Int("workers", settings.Workers).
Bool("dry_run", settings.DryRun).
Msg("Command settings parsed")
// Simulate processing
if settings.DryRun {
log.Info().Msg("Dry run mode - no actual processing")
} else {
log.Info().Msg("Starting actual data processing")
}
// Simulate some work with progress logging
for i := 0; i < settings.Workers; i++ {
log.Info().Int("worker_id", i).Msg("Starting worker")
// Simulate processing time
time.Sleep(100 * time.Millisecond)
// Create result row
row := types.NewRow(
types.MRP("worker_id", i),
types.MRP("status", "completed"),
types.MRP("processed_items", (i+1)*10),
types.MRP("duration_ms", 100),
types.MRP("timestamp", time.Now().Format(time.RFC3339)),
)
if err := gp.AddRow(ctx, row); err != nil {
log.Error().Err(err).Int("worker_id", i).Msg("Failed to add result row")
return err
}
log.Debug().Int("worker_id", i).Msg("Worker completed")
}
log.Info().Msg("Data processing completed successfully")
return nil
}
func NewProcessDataCommand() (*ProcessDataCommand, error) {
// Create logging section with custom options
loggingSection, err := logging.NewLoggingSectionWithOptions(
logging.WithDefaultLevel("info"),
logging.WithDefaultFormat("text"),
)
if err != nil {
return nil, err
}
// Create glazed section for output formatting
glazedSection, err := settings.NewGlazedSchema()
if err != nil {
return nil, err
}
cmdDesc := cmds.NewCommandDescription(
"process-data",
cmds.WithShort("Process data with configurable logging"),
cmds.WithLong(`
Process data files with comprehensive logging support.
This command demonstrates how to use the custom logging section.
Examples:
process-data --input-file data.csv --workers 4
process-data --input-file data.csv --log-level debug
process-data --input-file data.csv --log-format json --log-file process.log
process-data --input-file data.csv --verbose --with-caller
`),
cmds.WithFlags(
fields.New(
"input-file",
fields.TypeString,
fields.WithHelp("Input file to process"),
fields.WithRequired(true),
fields.WithShortFlag("i"),
),
fields.New(
"output-path",
fields.TypeString,
fields.WithHelp("Output file path"),
fields.WithDefault("output.processed"),
fields.WithShortFlag("o"),
),
fields.New(
"workers",
fields.TypeInteger,
fields.WithHelp("Number of worker processes"),
fields.WithDefault(2),
fields.WithShortFlag("w"),
),
fields.New(
"dry-run",
fields.TypeBool,
fields.WithHelp("Perform a dry run without actual processing"),
fields.WithDefault(false),
),
),
// Add both logging and glazed sections
cmds.WithSectionsList(loggingSection, glazedSection),
)
return &ProcessDataCommand{
CommandDescription: cmdDesc,
}, nil
}
var _ cmds.GlazeCommand = &ProcessDataCommand{}
// Second command to demonstrate section reuse
type AnalyzeDataCommand struct {
*cmds.CommandDescription
}
type AnalyzeDataSettings struct {
DataFile string `glazed:"data-file"`
Algorithm string `glazed:"algorithm"`
Iterations int `glazed:"iterations"`
}
func (c *AnalyzeDataCommand) RunIntoGlazeProcessor(
ctx context.Context,
parsedSections *values.Values,
gp middlewares.Processor,
) error {
// Initialize logging (same section, reused!)
if err := logging.InitializeLogging(parsedSections); err != nil {
return fmt.Errorf("failed to initialize logging: %w", err)
}
log.Info().Msg("Starting data analysis command")
settings := &AnalyzeDataSettings{}
if err := parsedSections.DecodeSectionInto(schema.DefaultSlug, settings); err != nil {
return err
}
log.Info().
Str("data_file", settings.DataFile).
Str("algorithm", settings.Algorithm).
Int("iterations", settings.Iterations).
Msg("Analysis configuration")
// Simulate analysis
for i := 0; i < settings.Iterations; i++ {
log.Debug().Int("iteration", i+1).Msg("Running analysis iteration")
// Simulate some analysis work
time.Sleep(50 * time.Millisecond)
row := types.NewRow(
types.MRP("iteration", i+1),
types.MRP("algorithm", settings.Algorithm),
types.MRP("accuracy", 0.85+float64(i)*0.01),
types.MRP("processing_time_ms", 50),
)
if err := gp.AddRow(ctx, row); err != nil {
return err
}
}
log.Info().Msg("Analysis completed")
return nil
}
func NewAnalyzeDataCommand() (*AnalyzeDataCommand, error) {
// Reuse the same logging section - this is the power of sections!
loggingSection, err := logging.NewLoggingSection()
if err != nil {
return nil, err
}
glazedSection, err := settings.NewGlazedSchema()
if err != nil {
return nil, err
}
cmdDesc := cmds.NewCommandDescription(
"analyze-data",
cmds.WithShort("Analyze data with configurable logging"),
cmds.WithLong("Analyze data files using various algorithms with the same logging configuration."),
cmds.WithFlags(
fields.New(
"data-file",
fields.TypeString,
fields.WithHelp("Data file to analyze"),
fields.WithRequired(true),
),
fields.New(
"algorithm",
fields.TypeChoice,
fields.WithChoices("linear", "logistic", "random-forest", "neural-net"),
fields.WithDefault("linear"),
fields.WithHelp("Analysis algorithm to use"),
),
fields.New(
"iterations",
fields.TypeInteger,
fields.WithDefault(3),
fields.WithHelp("Number of analysis iterations"),
),
),
cmds.WithSectionsList(loggingSection, glazedSection),
)
return &AnalyzeDataCommand{
CommandDescription: cmdDesc,
}, nil
}
var _ cmds.GlazeCommand = &AnalyzeDataCommand{}
func main() {
rootCmd := &cobra.Command{
Use: "data-processor",
Short: "Data processing application with custom logging section",
Long: "Demonstrates how to create and reuse custom field sections in Glazed",
}
// Create and register process command
processCmd, err := NewProcessDataCommand()
if err != nil {
fmt.Fprintf(os.Stderr, "Error creating process command: %v\n", err)
os.Exit(1)
}
cobraProcessCmd, err := cli.BuildCobraCommand(processCmd)
if err != nil {
fmt.Fprintf(os.Stderr, "Error building process command: %v\n", err)
os.Exit(1)
}
// Create and register analyze command
analyzeCmd, err := NewAnalyzeDataCommand()
if err != nil {
fmt.Fprintf(os.Stderr, "Error creating analyze command: %v\n", err)
os.Exit(1)
}
cobraAnalyzeCmd, err := cli.BuildCobraCommand(analyzeCmd)
if err != nil {
fmt.Fprintf(os.Stderr, "Error building analyze command: %v\n", err)
os.Exit(1)
}
rootCmd.AddCommand(cobraProcessCmd, cobraAnalyzeCmd)
if err := rootCmd.Execute(); err != nil {
os.Exit(1)
}
}
Build and test the application:
go build -o data-processor
# Test basic functionality
./data-processor process-data --help
./data-processor analyze-data --help
# Test different logging configurations
./data-processor process-data --input-file test.csv --workers 3
# Debug logging
./data-processor process-data --input-file test.csv --log-level debug
# JSON logging to file
./data-processor process-data --input-file test.csv --log-format json --log-file process.log
# Verbose mode with caller info
./data-processor process-data --input-file test.csv --verbose --with-caller
# Test analyze command (same logging options!)
./data-processor analyze-data --data-file test.csv --algorithm neural-net --log-level debug
# Combine with Glazed output options
./data-processor process-data --input-file test.csv --output json --fields worker_id,status,processed_items
Add environment variable support by using middleware when running commands:
// In your main.go, you could add:
import "github.com/go-go-golems/glazed/pkg/cmds/runner"
func runWithEnvironment() {
cmd, _ := NewProcessDataCommand()
parseOptions := []runner.ParseOption{
runner.WithEnvMiddleware("DATAPROC_"), // Loads DATAPROC_LOG_LEVEL, etc.
}
ctx := context.Background()
err := runner.ParseAndRun(ctx, cmd, parseOptions, nil)
if err != nil {
log.Fatal().Err(err).Msg("Command failed")
}
}
parseOptions := []runner.ParseOption{
runner.WithEnvMiddleware("DATAPROC_"),
runner.WithViper(), // Loads from config file
}
Create specialized sections by combining the logging section with others:
func NewDatabaseSectionWithLogging() ([]schema.Section, error) {
loggingSection, err := logging.NewLoggingSection()
if err != nil {
return nil, err
}
dbSection, err := database.NewDatabaseSection()
if err != nil {
return nil, err
}
return []schema.Section{loggingSection, dbSection}, nil
}
This tutorial demonstrates creating reusable configuration components that address common CLI development challenges.
Before sections: Adding logging to commands required copying flag definitions, validation logic, and initialization code across multiple files, leading to inconsistent behavior and maintenance overhead.
With sections: Adding logging to any command requires a single line: cmds.WithSectionsList(loggingSection). All commands share the same interface, validation, and behavior patterns.
Separation of Concerns: The logging section handles configuration independently from business logic, enabling isolated testing and reuse across different commands.
Early Validation: Validation methods catch configuration errors at startup rather than during runtime.
Sensible Defaults: The section provides working defaults for common use cases while supporting advanced configurations for enterprise requirements.
Convention Over Configuration: Consistent patterns for field naming, struct tags, and validation provide familiar interfaces for Go developers.
The implemented section includes production-ready capabilities:
Section composition enables modular architecture patterns for complex applications:
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ API Section │ │ Database Section │ │ Logging Section │
│ - base-url │ │ - db-host │ │ - log-level │
│ - timeout │ │ - db-port │ │ - log-format │
│ - retry-count │ │ - db-name │ │ - log-file │
│ - api-key │ │ - ssl-mode │ │ - verbose │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│ │ │
└─────────────────────┼─────────────────────┘
│
┌─────────▼─────────┐
│ Your Command │
│ - command-specific│
│ fields │
└───────────────────┘
Each section handles a specific concern. Commands compose required sections to build applications that scale from simple scripts to complex enterprise systems.
Common section implementations for production applications:
Database Section: Connection pooling, transaction management, migration flags
HTTP Client Section: Authentication, retries, circuit breakers, rate limiting
File Processing Section: Input/output directories, file patterns, validation
Cache Section: Redis configuration, TTL settings, eviction policies
Production systems benefit from shared base configurations:
// Base configuration shared across services
baseSection := NewBaseSection(
WithDefaultTimeout(30*time.Second),
WithDefaultRetries(3),
)
// Service-specific extensions
apiSection := NewAPISection(
WithAuthentication(),
WithRateLimiting(),
)
Enterprise section implementations must address:
type DatabaseSettings struct {
Host string `glazed:"db-host"`
Port int `glazed:"db-port"`
Name string `glazed:"db-name"`
Username string `glazed:"db-username"`
Password string `glazed:"db-password"`
SSLMode string `glazed:"db-ssl-mode"`
MaxConns int `glazed:"db-max-connections"`
MaxIdleTime string `glazed:"db-max-idle-time"`
}
type HTTPSettings struct {
BaseURL string `glazed:"base-url"`
Timeout time.Duration `glazed:"timeout"`
RetryCount int `glazed:"retry-count"`
RetryBackoff time.Duration `glazed:"retry-backoff"`
UserAgent string `glazed:"user-agent"`
APIKey string `glazed:"api-key"`
RateLimitRPS int `glazed:"rate-limit-rps"`
}
type FileSettings struct {
InputDir string `glazed:"input-dir"`
OutputDir string `glazed:"output-dir"`
FilePattern string `glazed:"file-pattern"`
Extensions []string `glazed:"extensions"`
Recursive bool `glazed:"recursive"`
OverwriteOK bool `glazed:"overwrite"`
BackupOld bool `glazed:"backup-existing"`
DryRun bool `glazed:"dry-run"`
}
This tutorial demonstrates implementing reusable field sections for CLI applications. The key principle is configuration through composition.
Rather than defining flags individually per command, standardized sections encapsulate interface and behavior patterns. This approach creates application consistency, reduces maintenance overhead, and provides predictable user interfaces.
The section pattern enables scalable CLI architecture that grows from simple commands to comprehensive enterprise applications.