freeDiameter: contrib/tools/csv_to

annotate contrib/tools/csv_to_fd @ 1506:2ab9d941bad5

csv_to_fd: improve validation error handling

author	Luke Mewburn <luke@mewburn.net>
date	Fri, 03 Apr 2020 17:04:13 +1100
parents	01efba83bf60
children	7c9214a5f095

rev	line source
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	1 #!/usr/bin/env python
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	2
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	3 """
1506 2ab9d941bad5 csv_to_fd: improve validation error handling Luke Mewburn <luke@mewburn.net> parents: 1498 diff changeset	4 Convert 8 column CSV files containing RADIUS or Diameter AVP tables
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	5 into various formats.
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	6
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	7 Format of the CSV files is one of:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	8 - Row per 3GPP AVP tables:
1506 2ab9d941bad5 csv_to_fd: improve validation error handling Luke Mewburn <luke@mewburn.net> parents: 1498 diff changeset	9 Name, Code, Section, DataType, Must, May, ShouldNot, MustNot
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	10 - Name:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	11 AVP Name. String, validated as ALPHA *(ALPHA / DIGIT / "-")
1463 8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	12 per RFC 6733 section 3.2.
1494 ae76ea63ee12 csv_to_fd: improve comments and usage Luke Mewburn <luke@mewburn.net> parents: 1487 diff changeset	13 May start with a DIGIT (e.g., "3GPP-IMSI").
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	14 - Code:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	15 AVP Code. Integer, 0..4294967295.
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	16 - Section:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	17 Section in relevant standard. String.
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	18 - DataType:
1463 8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	19 AVP Data Type. String, validated per basic and derived types in:
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	20 - RFC 6733 section 4.2
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	21 - RFC 6733 section 4.3
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	22 - RFC 7155 section 4.1
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	23 - Must, May, ShouldNot, MustNot:
1494 ae76ea63ee12 csv_to_fd: improve comments and usage Luke Mewburn <luke@mewburn.net> parents: 1487 diff changeset	24 Flags, possibly comma or space separated: M, P, V
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	25
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	26 - Comment row. First cell:
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	27 # Comment text 'Comment text'
c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	28 #= '/========/'
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	29 # Blank line
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	30
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	31 - Parameter row:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	32 @Parameter,Value [, ...]
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	33 Supported Parameter terms:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	34 standard Standard name. E.g. '3GPP TS 29.272', 'RFC 6733'.
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	35 vendor Vendor number.
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	36
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	37 """
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	38
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	39 from __future__ import print_function
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	40 from __future__ import with_statement
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	41
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	42 import abc
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	43 import csv
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	44 import collections
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	45 import json
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	46 import re
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	47 import optparse
1469 48fa8d70e6ad csv_to_fd: validate flags and other improvements Luke Mewburn <luke@mewburn.net> parents: 1464 diff changeset	48 import os
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	49 import sys
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	50
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	51 CSV_COLUMN_NAMES = [
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	52 'name',
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	53 'code',
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	54 'section',
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	55 'datatype',
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	56 'must',
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	57 'may',
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	58 'shouldnot',
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	59 'mustnot',
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	60 ]
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	61
1463 8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	62 DERIVED_TO_BASE = {
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	63 'Address': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	64 'Time': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	65 'UTF8String': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	66 'DiameterIdentity': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	67 'DiameterURI': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	68 'Enumerated': 'Integer32', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	69 'IPFilterRule': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	70 'QoSFilterRule': 'OctetString', # RFC 7155 section 4.1.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	71 }
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	72
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	73 # See https://www.iana.org/assignments/enterprise-numbers/enterprise-numbers
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	74 VENDOR_TO_NAME = {
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	75 0: '',
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	76 193: 'Ericsson',
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	77 8164: 'Starent',
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	78 10415: '3GPP',
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	79 }
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	80
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	81
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	82 class Avp(object):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	83 """Store an AVP row."""
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	84
1463 8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	85 # Regex to validate avp-name per RFC 6733 section 3.2,
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	86 # with changes:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	87 # - Allow avp-name to start with numbers (for 3GPP)
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	88 # - Allow '.' in avp-name, for existing dict_dcca_3gpp usage.
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	89 # TODO: if starts with digit, ensure contains a letter somewhere?
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	90 _name_re = re.compile(r'^[a-zA-Z0-9][a-zA-Z0-9-\.]*$')
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	91
1469 48fa8d70e6ad csv_to_fd: validate flags and other improvements Luke Mewburn <luke@mewburn.net> parents: 1464 diff changeset	92 # Regex to validate flags: M, P, V, comma, space
48fa8d70e6ad csv_to_fd: validate flags and other improvements Luke Mewburn <luke@mewburn.net> parents: 1464 diff changeset	93 _flags_re = re.compile(r'^[MPV, ]*$')
48fa8d70e6ad csv_to_fd: validate flags and other improvements Luke Mewburn <luke@mewburn.net> parents: 1464 diff changeset	94
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	95 __slots__ = CSV_COLUMN_NAMES + [
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	96 'filename', 'line_num', 'standard', 'vendor', ]
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	97
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	98 def __init__(self, name, code, section, datatype,
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	99 must, may, shouldnot, mustnot, extra_cells=[],
c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	100 filename='', line_num=0, standard='', vendor=0):
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	101 # Members from CSV row
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	102 self.name = name
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	103 self.code = int(code)
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	104 self.section = section
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	105 self.datatype = datatype
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	106 self.must = must
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	107 self.may = may
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	108 self.shouldnot = shouldnot
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	109 self.mustnot = mustnot
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	110 # Members from file state
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	111 self.filename = filename
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	112 self.line_num = line_num
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	113 self.standard = standard
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	114 self.vendor = vendor
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	115 # Validate CSV fields
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	116 if not self._name_re.match(self.name):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	117 raise ValueError('Invalid AVP name "{}"'.format(self.name))
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	118 if (self.code < 0 or self.code > 4294967295):
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	119 raise ValueError('AVP "{}" invalid code {}'.format(
c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	120 self.name, self.code))
1463 8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	121 if (self.datatype not in (
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	122 'OctetString', 'Integer32', 'Integer64', 'Unsigned32',
1463 8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	123 'Unsigned64', 'Float32', 'Float64', 'Grouped')
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	124 and self.datatype not in DERIVED_TO_BASE):
1487 5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	125 raise ValueError('{} invalid data type "{}"'.format(
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	126 self.description(), self.datatype))
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	127 # Validate flags
c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	128 flags = collections.Counter()
1469 48fa8d70e6ad csv_to_fd: validate flags and other improvements Luke Mewburn <luke@mewburn.net> parents: 1464 diff changeset	129 for val, desc in [
48fa8d70e6ad csv_to_fd: validate flags and other improvements Luke Mewburn <luke@mewburn.net> parents: 1464 diff changeset	130 (self.must, 'Must'),
48fa8d70e6ad csv_to_fd: validate flags and other improvements Luke Mewburn <luke@mewburn.net> parents: 1464 diff changeset	131 (self.may, 'May'),
48fa8d70e6ad csv_to_fd: validate flags and other improvements Luke Mewburn <luke@mewburn.net> parents: 1464 diff changeset	132 (self.shouldnot, 'Should Not'),
48fa8d70e6ad csv_to_fd: validate flags and other improvements Luke Mewburn <luke@mewburn.net> parents: 1464 diff changeset	133 (self.mustnot, 'Must Not'),
48fa8d70e6ad csv_to_fd: validate flags and other improvements Luke Mewburn <luke@mewburn.net> parents: 1464 diff changeset	134 ]:
48fa8d70e6ad csv_to_fd: validate flags and other improvements Luke Mewburn <luke@mewburn.net> parents: 1464 diff changeset	135 if not self._flags_re.match(val):
1487 5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	136 raise ValueError('{} invalid {} Flags "{}"'.format(
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	137 self.description(), desc, val))
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	138 flags.update(val)
c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	139 # Check occurrence of M,V in Must,May,ShouldNot,MustNot
c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	140 for flag in 'MV':
c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	141 # TODO: can AVP flags not appear at all?
c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	142 # if flags[flag] == 0:
1487 5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	143 # raise ValueError('{} Flag "{}" not set'.format(
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	144 # self.description(), flag))
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	145 if flags[flag] > 1:
1487 5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	146 raise ValueError('{} Flag "{}" set {} times'.format(
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	147 self.description(), flag, flags[flag]))
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	148 # Compare V presence against vendor
c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	149 if 'V' in self.must:
c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	150 if self.vendor == 0:
1487 5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	151 raise ValueError('{} Flag "V" set for vendor 0'.format(
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	152 self.description()))
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	153 else:
c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	154 if self.vendor != 0:
1487 5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	155 raise ValueError('{} Flag "V" not set for vendor {}'.format(
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	156 self.description(), self.vendor))
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	157
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	158 @property
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	159 def __dict__(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	160 return {s: getattr(self, s) for s in self.__slots__}
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	161
1487 5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	162 def __eq__(self, other):
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	163 """Equality comparison of Avp instances.
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	164 Considered equal if name, vendor, code, datatype, and flags are equal.
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	165 """
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	166 if other is self:
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	167 return True
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	168 if type(other) is not type(self):
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	169 return NotImplemented
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	170 return (
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	171 other.name, other.vendor, other.code, other.datatype,
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	172 other.must, other.may, other.shouldnot, other.mustnot,
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	173 ) == (
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	174 self.name, self.vendor, self.code, self.datatype,
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	175 self.must, self.may, self.shouldnot, self.mustnot,
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	176 )
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	177
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	178 def __ne__(self, other):
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	179 return not self == other
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	180
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	181 def description(self):
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	182 return 'AVP "{}" ({})'.format(self.name, self.code)
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	183
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	184
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	185 class Processor(object):
1494 ae76ea63ee12 csv_to_fd: improve comments and usage Luke Mewburn <luke@mewburn.net> parents: 1487 diff changeset	186 """Interface for processor of Avp."""
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	187
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	188 __metaclass__ = abc.ABCMeta
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	189
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	190 @classmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	191 def cls_name(cls):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	192 """Return the name, lower-case, without "processor" suffix."""
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	193 suffix = 'processor'
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	194 name = cls.__name__.lower()
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	195 if name.endswith(suffix):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	196 return name[:-len(suffix)]
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	197 return name
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	198
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	199 @classmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	200 def cls_desc(cls):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	201 """Return the first line of the docstring."""
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	202 if cls.__doc__ is None:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	203 return ""
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	204 return cls.__doc__.split('\n')[0]
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	205
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	206 @abc.abstractmethod
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	207 def filename(self, filename):
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	208 """Called when a file is opened."""
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	209 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	210
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	211 @abc.abstractmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	212 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	213 """Process a validated Avp."""
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	214 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	215
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	216 @abc.abstractmethod
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	217 def comment(self, comment, filename, line_num):
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	218 """Process a comment row:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	219 #comment,
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	220 """
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	221 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	222
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	223 @abc.abstractmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	224 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	225 """Invoked after all rows processed."""
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	226 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	227
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	228 @abc.abstractmethod
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	229 def parameter(self, name, value):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	230 """Process a parameter row:
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	231 @name,value.
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	232 """
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	233 pass
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	234
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	235
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	236 class DebugProcessor(Processor):
1494 ae76ea63ee12 csv_to_fd: improve comments and usage Luke Mewburn <luke@mewburn.net> parents: 1487 diff changeset	237 """Display the CSV parsing."""
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	238
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	239 def filename(self, filename):
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	240 print('File: {}'.format(filename))
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	241
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	242 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	243 avpdict = vars(avp)
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	244 print('AVP: {name}, {code}, {datatype}'.format(**avpdict))
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	245
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	246 def comment(self, comment, filename, line_num):
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	247 print('Comment: {}'.format(comment))
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	248
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	249 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	250 print('Generate')
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	251
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	252 def parameter(self, name, value):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	253 print('Parameter: {} {}'.format(name, value))
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	254
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	255
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	256 class NoopProcessor(Processor):
1494 ae76ea63ee12 csv_to_fd: improve comments and usage Luke Mewburn <luke@mewburn.net> parents: 1487 diff changeset	257 """Validate the CSV; no other output."""
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	258
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	259 def filename(self, filename):
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	260 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	261
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	262 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	263 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	264
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	265 def comment(self, comment, filename, line_num):
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	266 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	267
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	268 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	269 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	270
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	271 def parameter(self, name, value):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	272 pass
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	273
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	274
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	275 class FdcProcessor(Processor):
1494 ae76ea63ee12 csv_to_fd: improve comments and usage Luke Mewburn <luke@mewburn.net> parents: 1487 diff changeset	276 """Generate freeDiameter C code.
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	277
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	278 Comment cells are parsed as:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	279 # text comment /* text comment */
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	280 #= /==============/
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	281 # [blank line]
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	282 """
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	283
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	284 COMMENT_WIDTH = 64
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	285
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	286 class AvpFunction(object):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	287 """Maintain per-function state to create DICT_AVP entries.
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	288 """
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	289
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	290 def __init__(self, name):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	291 self.__name = name
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	292 self.__lines = []
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	293 self.__derived = set()
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	294
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	295 @property
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	296 def name(self):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	297 """Return name."""
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	298 return self.__name
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	299
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	300 @property
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	301 def lines(self):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	302 """Return all lines."""
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	303 return self.__lines
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	304
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	305 @lines.setter
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	306 def lines(self, value):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	307 """Set to append a line."""
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	308 self.__lines.append(value)
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	309
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	310 @property
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	311 def derived(self):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	312 """Return list of all derived values."""
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	313 return list(self.__derived)
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	314
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	315 @derived.setter
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	316 def derived(self, value):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	317 """Set to store a derived type."""
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	318 self.__derived.add(value)
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	319
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	320 def __init__(self):
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	321 self._filenames = []
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	322 self._functions = collections.OrderedDict()
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	323
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	324 def filename(self, filename):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	325 self._filenames.append(os.path.basename(filename))
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	326
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	327 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	328 comment = '{name}, {datatype}, code {code}'.format(**vars(avp))
1464 1404de313b85 csv_to_fd: consistent comparison order Luke Mewburn <luke@mewburn.net> parents: 1463 diff changeset	329 if avp.section != '':
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	330 comment += ', section {}'.format(avp.section)
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	331 self.add_comment(comment)
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	332 self.add('\t{')
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	333 self.add('\t\tstruct dict_avp_data data = {')
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	334 # TODO: remove comments?
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	335 self.add('\t\t\t{},\t/* Code */'.format(avp.code))
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	336 self.add('\t\t\t{},\t/* Vendor */'.format(avp.vendor))
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	337 self.add('\t\t\t\"{}\",\t/* Name */'.format(avp.name))
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	338 self.add('\t\t\t{},\t/* Fixed flags */'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	339 self.build_flags(', '.join([avp.must, avp.mustnot]))))
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	340 self.add('\t\t\t{},\t/* Fixed flag values */'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	341 self.build_flags(avp.must)))
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	342 # TODO: add trailing comma?
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	343 self.add('\t\t\tAVP_TYPE_{}\t/* base type of data */'.format(
1463 8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	344 DERIVED_TO_BASE.get(avp.datatype, avp.datatype).upper()))
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	345 self.add('\t\t};')
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	346 avp_type = 'NULL'
1464 1404de313b85 csv_to_fd: consistent comparison order Luke Mewburn <luke@mewburn.net> parents: 1463 diff changeset	347 if avp.datatype == 'Enumerated':
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	348 self.add('\t\tstruct dict_object\t*type;')
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	349 vendor_prefix = ''
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	350 if avp.vendor != 0:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	351 vendor_prefix = '{}/'.format(VENDOR_TO_NAME[avp.vendor])
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	352 self.add(
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	353 '\t\tstruct dict_type_data\t tdata = {{ AVP_TYPE_INTEGER32, '
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	354 '"Enumerated({prefix}{name})", NULL, NULL, NULL }};'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	355 prefix=vendor_prefix, name=avp.name))
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	356 # XXX: add enumerated values
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	357 self.add('\t\tCHECK_dict_new(DICT_TYPE, &tdata, NULL, &type);')
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	358 avp_type = "type"
1463 8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes Luke Mewburn <luke@mewburn.net> parents: 1461 diff changeset	359 elif avp.datatype in DERIVED_TO_BASE:
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	360 avp_type = '{}_type'.format(avp.datatype)
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	361 self.derived(avp.datatype)
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	362 self.add('\t\tCHECK_dict_new(DICT_AVP, &data, {}, NULL);'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	363 avp_type))
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	364 # TODO: remove ; on scope brace
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	365 self.add('\t};')
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	366 self.add('')
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	367
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	368 def comment(self, comment, filename, line_num):
1464 1404de313b85 csv_to_fd: consistent comparison order Luke Mewburn <luke@mewburn.net> parents: 1463 diff changeset	369 if comment == '':
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	370 self.add('')
1464 1404de313b85 csv_to_fd: consistent comparison order Luke Mewburn <luke@mewburn.net> parents: 1463 diff changeset	371 elif comment == '=':
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	372 self.add_header()
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	373 elif comment.startswith(' '):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	374 self.add_comment(comment[1:])
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	375 else:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	376 raise ValueError('Unsupported comment "{}"'.format(comment))
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	377
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	378 def generate(self):
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	379 fp = sys.stdout
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	380 self.write_introduction(fp)
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	381 for func in self._functions:
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	382 self.write_function(fp, self._functions[func])
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	383
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	384 def parameter(self, name, value):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	385 pass
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	386
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	387 # internal methods
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	388
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	389 def current_avpfunction(self):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	390 """Return current AvpFunction to update.
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	391
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	392 Note: allows for easier future enhancement to generate separate
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	393 C functions per AVP groups such as: by csv file, standard, or vendor.
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	394 """
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	395 name = 'add_avps'
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	396 if name not in self._functions:
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	397 self._functions[name] = self.AvpFunction(name)
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	398 return self._functions[name]
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	399
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	400 def add(self, line):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	401 self.current_avpfunction().lines = line
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	402
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	403 def derived(self, value):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	404 self.current_avpfunction().derived = value
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	405
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	406 def build_c_token(self, value):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	407 """Convert a string into a valid C token."""
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	408 return re.sub(r'[^\w]', '_', value)
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	409
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	410 def build_flags(self, flags):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	411 result = []
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	412 if 'V' in flags:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	413 result.append('AVP_FLAG_VENDOR')
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	414 if 'M' in flags:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	415 result.append('AVP_FLAG_MANDATORY')
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	416 return ' \|'.join(result)
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	417
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	418 def add_comment(self, comment):
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	419 self.add(self.format_comment(comment))
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	420
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	421 def add_header(self):
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	422 self.add(self.format_header())
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	423
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	424 def format_comment(self, comment):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	425 return '\t/* {:<{width}} */'.format(comment, width=self.COMMENT_WIDTH)
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	426
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	427 def format_header(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	428 return '\t/={:=<{width}}=/'.format('', width=self.COMMENT_WIDTH)
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	429
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	430 def write_introduction(self, fp):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	431 """Write the introduction to the generated file."""
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	432 fp.write('''\
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	433 /*
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	434 Generated by:
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	435 \tcsv_to_fd -p {processor} {files}
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	436
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	437 Do not modify; modify the source .csv files instead.
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	438 */
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	439
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	440 #include <freeDiameter/extension.h>
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	441
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	442 #define CHECK_dict_new( _type, _data, _parent, _ref ) \\
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	443 \tCHECK_FCT( fd_dict_new( fd_g_config->cnf_dict, \
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	444 (_type), (_data), (_parent), (_ref)) );
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	445
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	446 #define CHECK_dict_search( _type, _criteria, _what, _result ) \\
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	447 \tCHECK_FCT( fd_dict_search( fd_g_config->cnf_dict, \
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	448 (_type), (_criteria), (_what), (_result), ENOENT) );
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	449 '''.format(
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	450 processor=self.cls_name(),
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	451 files=' '.join(self._filenames)))
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	452
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	453 def write_function(self, fp, avpfunction):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	454 """Generate a function from AvpFunction."""
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	455 function = self.build_c_token(avpfunction.name)
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	456 # Function start
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	457 fp.write('''\
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	458
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	459 int {}()
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	460 {{
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	461 '''.format(function))
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	462
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	463 # Create variables used by derived type validation
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	464 for derived in avpfunction.derived:
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	465 fp.write('''\
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	466 \tstruct dict_object * {name}_type = NULL;
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	467 \tCHECK_dict_search(DICT_TYPE, TYPE_BY_NAME, "{name}", &{name}_type);
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	468
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	469 '''.format(name=derived))
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	470
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	471 # Write generated DICT_AVP creation
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	472 fp.write('\n'.join(avpfunction.lines))
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	473
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	474 # Write function end
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	475 fp.write('''\
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	476
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	477 \treturn 0;
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	478 }} /* {}() */
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	479 '''.format(function))
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	480
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	481
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	482 class JsonProcessor(Processor):
1494 ae76ea63ee12 csv_to_fd: improve comments and usage Luke Mewburn <luke@mewburn.net> parents: 1487 diff changeset	483 """Generate freeDiameter JSON object.
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	484 """
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	485
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	486 def __init__(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	487 self.avps = []
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	488
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	489 def filename(self, filename):
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	490 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	491
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	492 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	493 flags = collections.OrderedDict([
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	494 ('Must', self.build_flags(avp.must)),
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	495 ('MustNot', self.build_flags(avp.mustnot)),
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	496 ])
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	497 row = collections.OrderedDict([
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	498 ('Code', avp.code),
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	499 ('Flags', flags),
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	500 ('Name', avp.name),
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	501 ('Type', avp.datatype),
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	502 ('Vendor', avp.vendor),
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	503 ])
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	504 self.avps.append(row)
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	505
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	506 def comment(self, comment, filename, line_num):
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	507 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	508
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	509 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	510 doc = {"AVPs": self.avps}
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	511 print(json.dumps(doc, indent=2))
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	512
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	513 def parameter(self, name, value):
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	514 pass
01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	515
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	516 def build_flags(self, flags):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	517 result = []
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	518 if 'V' in flags:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	519 result.append('V')
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	520 if 'M' in flags:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	521 result.append('M')
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	522 return ''.join(result)
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	523
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	524
1487 5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	525 def avp_conflict(description, avp, conflict):
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	526 """Raise error for duplicate or conflicting AVPs.
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	527 """
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	528 if avp == conflict:
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	529 raise ValueError(
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	530 '{} {} duplicated in'
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	531 ' file "{}" line {}'.format(
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	532 avp.description(), description,
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	533 conflict.filename, conflict.line_num))
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	534 else:
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	535 raise ValueError(
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	536 '{} {} conflicts with {}'
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	537 ' in file "{}" line {}'.format(
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	538 avp.description(), description,
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	539 conflict.description(),
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	540 conflict.filename, conflict.line_num))
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	541
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	542
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	543 def main():
1487 5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	544 """Main application entry.
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	545 """
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	546
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	547 # Build dict of name: NameProcessor
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	548 processors = {
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	549 cls.cls_name(): cls
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	550 for cls in Processor.__subclasses__()
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	551 }
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	552
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	553 # Build Processor name to desc
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	554 processor_help = '\n'.join(
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	555 [' {:8} {}'.format(key, processors[key].cls_desc())
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	556 for key in sorted(processors)])
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	557
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	558 # Custom OptionParser with improved help
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	559 class MyParser(optparse.OptionParser):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	560 """Custom OptionParser without epilog formatting."""
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	561 def format_help(self, formatter=None):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	562 return """\
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	563 {}
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	564 Supported PROCESSOR options:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	565 {}
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	566 """.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	567 optparse.OptionParser.format_help(self, formatter),
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	568 processor_help)
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	569
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	570 # Parse options
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	571 parser = MyParser(
1494 ae76ea63ee12 csv_to_fd: improve comments and usage Luke Mewburn <luke@mewburn.net> parents: 1487 diff changeset	572 usage='%prog [-h] [-p PROCESSOR] FILE ...',
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	573 description="""\
1494 ae76ea63ee12 csv_to_fd: improve comments and usage Luke Mewburn <luke@mewburn.net> parents: 1487 diff changeset	574 Convert CSV files FILE ... containing RADIUS or Diameter AVP tables
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	575 into various formats using the specified processor PROCESSOR.
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	576 """)
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	577 parser.add_option(
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	578 '-p', '--processor',
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	579 default='noop',
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	580 help='AVP processor. One of: {}. [%default]'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	581 ', '.join(processors.keys())))
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	582 (opts, args) = parser.parse_args()
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	583 if len(args) < 1:
1494 ae76ea63ee12 csv_to_fd: improve comments and usage Luke Mewburn <luke@mewburn.net> parents: 1487 diff changeset	584 parser.error('Incorrect number of arguments. Use -h for help.')
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	585
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	586 # Find processor
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	587 try:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	588 avpproc = processors[opts.processor]()
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	589 except KeyError as e:
1494 ae76ea63ee12 csv_to_fd: improve comments and usage Luke Mewburn <luke@mewburn.net> parents: 1487 diff changeset	590 parser.error('Unknown processor "{}".'.format(opts.processor))
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	591
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	592 # dict of [vendor][code] : Avp
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	593 avp_codes = collections.defaultdict(dict)
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	594
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	595 # dict of [vendor][name] : Avp
c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	596 avp_names = collections.defaultdict(dict)
c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	597
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	598 # Process files
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	599 for filename in args:
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	600 avpproc.filename(filename)
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	601 with open(filename, 'r') as csvfile:
1469 48fa8d70e6ad csv_to_fd: validate flags and other improvements Luke Mewburn <luke@mewburn.net> parents: 1464 diff changeset	602 csvdata = csv.DictReader(csvfile, CSV_COLUMN_NAMES,
1506 2ab9d941bad5 csv_to_fd: improve validation error handling Luke Mewburn <luke@mewburn.net> parents: 1498 diff changeset	603 restkey='extra_cells', restval='')
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	604 standard = ''
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	605 vendor = 0
1487 5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	606 errors = []
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	607 for row in csvdata:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	608 try:
1506 2ab9d941bad5 csv_to_fd: improve validation error handling Luke Mewburn <luke@mewburn.net> parents: 1498 diff changeset	609 if csvdata.restkey in row:
2ab9d941bad5 csv_to_fd: improve validation error handling Luke Mewburn <luke@mewburn.net> parents: 1498 diff changeset	610 raise ValueError('Extra cells: {}'.format(
2ab9d941bad5 csv_to_fd: improve validation error handling Luke Mewburn <luke@mewburn.net> parents: 1498 diff changeset	611 ','.join(row[csvdata.restkey])))
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	612 if row['name'] in (None, '', 'Attribute Name'):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	613 continue
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	614 elif row['name'].startswith('#'):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	615 comment = row['name'][1:]
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	616 avpproc.comment(comment, filename, csvdata.line_num)
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	617 elif row['name'].startswith('@'):
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	618 parameter = row['name'][1:]
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	619 value = row['code']
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	620 if False:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	621 pass
1464 1404de313b85 csv_to_fd: consistent comparison order Luke Mewburn <luke@mewburn.net> parents: 1463 diff changeset	622 elif parameter == 'standard':
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	623 standard = value
1464 1404de313b85 csv_to_fd: consistent comparison order Luke Mewburn <luke@mewburn.net> parents: 1463 diff changeset	624 elif parameter == 'vendor':
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	625 vendor = int(value)
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	626 else:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	627 raise ValueError('Unknown parameter "{}"'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	628 parameter))
1498 01efba83bf60 csv_to_fd: fdc generate complete file Luke Mewburn <luke@mewburn.net> parents: 1494 diff changeset	629 avpproc.parameter(parameter, value)
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	630 else:
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	631 avp = Avp(filename=filename, line_num=csvdata.line_num,
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	632 standard=standard, vendor=vendor,
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	633 **row)
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	634 # Ensure AVP vendor/code not already defined
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	635 if avp.code in avp_codes[avp.vendor]:
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	636 conflict = avp_codes[avp.vendor][avp.code]
1487 5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	637 avp_conflict('Code', avp, conflict)
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	638 avp_codes[avp.vendor][avp.code] = avp
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	639 # Ensure AVP vendor/name not already defined
c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	640 if avp.name in avp_names[avp.vendor]:
c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	641 conflict = avp_names[avp.vendor][avp.name]
1487 5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	642 avp_conflict('Name', avp, conflict)
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	643 avp_names[avp.vendor][avp.name] = avp
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	644 # Process AVP
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	645 avpproc.avp(avp)
1506 2ab9d941bad5 csv_to_fd: improve validation error handling Luke Mewburn <luke@mewburn.net> parents: 1498 diff changeset	646 except (TypeError, ValueError) as e:
1487 5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	647 errors.append('CSV file "{}" line {}: {}\n'.format(
1479 c0aa1e66c12e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1469 diff changeset	648 filename, csvdata.line_num, e))
1487 5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	649 if errors:
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	650 sys.stderr.write(''.join(errors))
5c2d061a8c8e csv_to_fd: improve validation Luke Mewburn <luke@mewburn.net> parents: 1479 diff changeset	651 sys.exit(1)
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	652
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	653 # Generate result
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	654 avpproc.generate()
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	655
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	656
1464 1404de313b85 csv_to_fd: consistent comparison order Luke Mewburn <luke@mewburn.net> parents: 1463 diff changeset	657 if __name__ == '__main__':
1461 a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	658 main()
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	659
a86eb3375b95 add csv_to_fd, org_to_csv tools Luke Mewburn <luke@mewburn.net> parents: diff changeset	660 # vim: set et sw=4 sts=4 :

Mercurial > hg > freeDiameter

annotate contrib/tools/csv_to_fd @ 1506:2ab9d941bad5