Mercurial > hg > freeDiameter

annotate contrib/tools/csv_to_fd @ 1494:ae76ea63ee12

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
csv_to_fd: improve comments and usage
author Luke Mewburn <luke@mewburn.net>
date Mon, 30 Mar 2020 16:46:31 +1100
parents 5c2d061a8c8e
children 01efba83bf60
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
1 #!/usr/bin/env python
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
2
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
3 """
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
4 Convert CSV files containing RADIUS or Diameter AVP tables
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
5 into various formats.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
6
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
7 Format of the CSV files is one of:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
8 - Row per 3GPP AVP tables:
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
9 Name, Code, Section, DataType, Must, May, ShouldNot, MustNot [, extra]
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
10 - Name:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
11 AVP Name. String, validated as ALPHA *(ALPHA / DIGIT / "-")
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
12 per RFC 6733 section 3.2.
1494
ae76ea63ee12 csv_to_fd: improve comments and usage
Luke Mewburn <luke@mewburn.net>
parents: 1487
diff changeset
13 May start with a DIGIT (e.g., "3GPP-IMSI").
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
14 - Code:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
15 AVP Code. Integer, 0..4294967295.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
16 - Section:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
17 Section in relevant standard. String.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
18 - DataType:
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
19 AVP Data Type. String, validated per basic and derived types in:
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
20 - RFC 6733 section 4.2
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
21 - RFC 6733 section 4.3
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
22 - RFC 7155 section 4.1
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
23 - Must, May, ShouldNot, MustNot:
1494
ae76ea63ee12 csv_to_fd: improve comments and usage
Luke Mewburn <luke@mewburn.net>
parents: 1487
diff changeset
24 Flags, possibly comma or space separated: M, P, V
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
25
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
26 - Comment row. First cell:
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
27 # Comment text 'Comment text'
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
28 #= '/*========*/'
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
29 # Blank line
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
30
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
31 - Parameter row:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
32 @Parameter,Value [, ...]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
33 Supported Parameter terms:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
34 standard Standard name. E.g. '3GPP TS 29.272', 'RFC 6733'.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
35 vendor Vendor number.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
36
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
37 """
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
38
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
39 from __future__ import print_function
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
40 from __future__ import with_statement
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
41
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
42 import abc
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
43 import csv
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
44 import collections
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
45 import json
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
46 import re
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
47 import optparse
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
48 import os
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
49 import sys
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
50
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
51 CSV_COLUMN_NAMES = [
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
52 'name',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
53 'code',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
54 'section',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
55 'datatype',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
56 'must',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
57 'may',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
58 'shouldnot',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
59 'mustnot',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
60 ]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
61
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
62 DERIVED_TO_BASE = {
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
63 'Address': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
64 'Time': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
65 'UTF8String': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
66 'DiameterIdentity': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
67 'DiameterURI': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
68 'Enumerated': 'Integer32', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
69 'IPFilterRule': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
70 'QoSFilterRule': 'OctetString', # RFC 7155 section 4.1.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
71 }
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
72
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
73 # See https://www.iana.org/assignments/enterprise-numbers/enterprise-numbers
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
74 VENDOR_TO_NAME = {
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
75 0: '',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
76 193: 'Ericsson',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
77 8164: 'Starent',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
78 10415: '3GPP',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
79 }
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
80
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
81
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
82 class Avp(object):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
83 """Store an AVP row."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
84
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
85 # Regex to validate avp-name per RFC 6733 section 3.2,
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
86 # with changes:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
87 # - Allow avp-name to start with numbers (for 3GPP)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
88 # - Allow '.' in avp-name, for existing dict_dcca_3gpp usage.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
89 # TODO: if starts with digit, ensure contains a letter somewhere?
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
90 _name_re = re.compile(r'^[a-zA-Z0-9][a-zA-Z0-9-\.]*$')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
91
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
92 # Regex to validate flags: M, P, V, comma, space
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
93 _flags_re = re.compile(r'^[MPV, ]*$')
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
94
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
95 __slots__ = CSV_COLUMN_NAMES + [
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
96 'filename', 'line_num', 'standard', 'vendor', ]
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
97
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
98 def __init__(self, name, code, section, datatype,
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
99 must, may, shouldnot, mustnot, extra_cells=[],
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
100 filename='', line_num=0, standard='', vendor=0):
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
101 # Members from CSV row
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
102 self.name = name
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
103 self.code = int(code)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
104 self.section = section
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
105 self.datatype = datatype
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
106 self.must = must
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
107 self.may = may
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
108 self.shouldnot = shouldnot
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
109 self.mustnot = mustnot
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
110 # Members from file state
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
111 self.filename = filename
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
112 self.line_num = line_num
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
113 self.standard = standard
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
114 self.vendor = vendor
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
115 # Validate CSV fields
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
116 if not self._name_re.match(self.name):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
117 raise ValueError('Invalid AVP name "{}"'.format(self.name))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
118 if (self.code < 0 or self.code > 4294967295):
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
119 raise ValueError('AVP "{}" invalid code {}'.format(
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
120 self.name, self.code))
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
121 if (self.datatype not in (
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
122 'OctetString', 'Integer32', 'Integer64', 'Unsigned32',
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
123 'Unsigned64', 'Float32', 'Float64', 'Grouped')
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
124 and self.datatype not in DERIVED_TO_BASE):
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
125 raise ValueError('{} invalid data type "{}"'.format(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
126 self.description(), self.datatype))
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
127 # Validate flags
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
128 flags = collections.Counter()
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
129 for val, desc in [
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
130 (self.must, 'Must'),
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
131 (self.may, 'May'),
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
132 (self.shouldnot, 'Should Not'),
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
133 (self.mustnot, 'Must Not'),
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
134 ]:
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
135 if not self._flags_re.match(val):
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
136 raise ValueError('{} invalid {} Flags "{}"'.format(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
137 self.description(), desc, val))
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
138 flags.update(val)
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
139 # Check occurrence of M,V in Must,May,ShouldNot,MustNot
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
140 for flag in 'MV':
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
141 # TODO: can AVP flags not appear at all?
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
142 # if flags[flag] == 0:
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
143 # raise ValueError('{} Flag "{}" not set'.format(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
144 # self.description(), flag))
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
145 if flags[flag] > 1:
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
146 raise ValueError('{} Flag "{}" set {} times'.format(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
147 self.description(), flag, flags[flag]))
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
148 # Compare V presence against vendor
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
149 if 'V' in self.must:
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
150 if self.vendor == 0:
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
151 raise ValueError('{} Flag "V" set for vendor 0'.format(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
152 self.description()))
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
153 else:
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
154 if self.vendor != 0:
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
155 raise ValueError('{} Flag "V" not set for vendor {}'.format(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
156 self.description(), self.vendor))
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
157
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
158 @property
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
159 def __dict__(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
160 return {s: getattr(self, s) for s in self.__slots__}
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
161
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
162 def __eq__(self, other):
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
163 """Equality comparison of Avp instances.
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
164 Considered equal if name, vendor, code, datatype, and flags are equal.
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
165 """
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
166 if other is self:
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
167 return True
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
168 if type(other) is not type(self):
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
169 return NotImplemented
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
170 return (
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
171 other.name, other.vendor, other.code, other.datatype,
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
172 other.must, other.may, other.shouldnot, other.mustnot,
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
173 ) == (
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
174 self.name, self.vendor, self.code, self.datatype,
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
175 self.must, self.may, self.shouldnot, self.mustnot,
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
176 )
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
177
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
178 def __ne__(self, other):
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
179 return not self == other
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
180
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
181 def description(self):
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
182 return 'AVP "{}" ({})'.format(self.name, self.code)
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
183
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
184
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
185 class Processor(object):
1494
ae76ea63ee12 csv_to_fd: improve comments and usage
Luke Mewburn <luke@mewburn.net>
parents: 1487
diff changeset
186 """Interface for processor of Avp."""
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
187
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
188 __metaclass__ = abc.ABCMeta
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
189
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
190 @classmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
191 def cls_name(cls):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
192 """Return the name, lower-case, without "processor" suffix."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
193 suffix = 'processor'
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
194 name = cls.__name__.lower()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
195 if name.endswith(suffix):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
196 return name[:-len(suffix)]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
197 return name
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
198
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
199 @classmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
200 def cls_desc(cls):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
201 """Return the first line of the docstring."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
202 if cls.__doc__ is None:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
203 return ""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
204 return cls.__doc__.split('\n')[0]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
205
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
206 @abc.abstractmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
207 def next_file(self, filename):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
208 """Called when a file is opened."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
209 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
210
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
211 @abc.abstractmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
212 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
213 """Process a validated Avp."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
214 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
215
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
216 @abc.abstractmethod
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
217 def comment(self, comment, filename, line_num):
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
218 """Process a comment row:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
219 #comment,
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
220 """
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
221 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
222
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
223 @abc.abstractmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
224 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
225 """Invoked after all rows processed."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
226 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
227
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
228
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
229 class DebugProcessor(Processor):
1494
ae76ea63ee12 csv_to_fd: improve comments and usage
Luke Mewburn <luke@mewburn.net>
parents: 1487
diff changeset
230 """Display the CSV parsing."""
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
231
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
232 def next_file(self, filename):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
233 print('File: {}'.format(filename))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
234
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
235 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
236 avpdict = vars(avp)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
237 print('AVP: {name}, {code}, {datatype}'.format(**avpdict))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
238
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
239 def comment(self, comment, filename, line_num):
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
240 print('Comment: {}'.format(comment))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
241
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
242 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
243 print('Generate')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
244
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
245
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
246 class NoopProcessor(Processor):
1494
ae76ea63ee12 csv_to_fd: improve comments and usage
Luke Mewburn <luke@mewburn.net>
parents: 1487
diff changeset
247 """Validate the CSV; no other output."""
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
248
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
249 def next_file(self, filename):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
250 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
251
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
252 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
253 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
254
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
255 def comment(self, comment, filename, line_num):
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
256 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
257
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
258 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
259 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
260
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
261
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
262 class FdcProcessor(Processor):
1494
ae76ea63ee12 csv_to_fd: improve comments and usage
Luke Mewburn <luke@mewburn.net>
parents: 1487
diff changeset
263 """Generate freeDiameter C code.
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
264
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
265 Comment cells are parsed as:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
266 # text comment /* text comment */
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
267 #= /*==============*/
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
268 # [blank line]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
269 """
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
270
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
271 COMMENT_WIDTH = 64
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
272
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
273 def __init__(self):
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
274 self.filenames = []
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
275 self.lines = []
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
276
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
277 def next_file(self, filename):
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
278 self.filenames.append(os.path.basename(filename))
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
279
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
280 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
281 comment = '{name}, {datatype}, code {code}'.format(**vars(avp))
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
282 if avp.section != '':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
283 comment += ', section {}'.format(avp.section)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
284 self.add_comment(comment)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
285 self.add('\t{')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
286 self.add('\t\tstruct dict_avp_data data = {')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
287 # TODO: remove comments?
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
288 self.add('\t\t\t{},\t/* Code */'.format(avp.code))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
289 self.add('\t\t\t{},\t/* Vendor */'.format(avp.vendor))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
290 self.add('\t\t\t\"{}\",\t/* Name */'.format(avp.name))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
291 self.add('\t\t\t{},\t/* Fixed flags */'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
292 self.build_flags(', '.join([avp.must, avp.mustnot]))))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
293 self.add('\t\t\t{},\t/* Fixed flag values */'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
294 self.build_flags(avp.must)))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
295 # TODO: add trailing comma?
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
296 self.add('\t\t\tAVP_TYPE_{}\t/* base type of data */'.format(
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
297 DERIVED_TO_BASE.get(avp.datatype, avp.datatype).upper()))
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
298 self.add('\t\t};')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
299 avp_type = 'NULL'
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
300 if avp.datatype == 'Enumerated':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
301 self.add('\t\tstruct dict_object\t*type;')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
302 vendor_prefix = ''
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
303 if avp.vendor != 0:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
304 vendor_prefix = '{}/'.format(VENDOR_TO_NAME[avp.vendor])
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
305 self.add(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
306 '\t\tstruct dict_type_data\t tdata = {{ AVP_TYPE_INTEGER32, '
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
307 '"Enumerated({prefix}{name})", NULL, NULL, NULL }};'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
308 prefix=vendor_prefix, name=avp.name))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
309 # XXX: add enumerated values
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
310 self.add('\t\tCHECK_dict_new(DICT_TYPE, &tdata, NULL, &type);')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
311 avp_type = "type"
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
312 elif avp.datatype in DERIVED_TO_BASE:
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
313 avp_type = '{}_type'.format(avp.datatype)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
314 self.add('\t\tCHECK_dict_new(DICT_AVP, &data, {}, NULL);'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
315 avp_type))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
316 # TODO: remove ; on scope brace
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
317 self.add('\t};')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
318 self.add('')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
319
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
320 def comment(self, comment, filename, line_num):
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
321 if comment == '':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
322 self.add('')
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
323 elif comment == '=':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
324 self.add_header()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
325 elif comment.startswith(' '):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
326 self.add_comment(comment[1:])
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
327 else:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
328 raise ValueError('Unsupported comment "{}"'.format(comment))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
329
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
330 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
331 self.print_header()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
332 self.print_comment('Start of generated data.')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
333 self.print_comment('')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
334 self.print_comment('The following is created automatically with:')
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
335 self.print_comment(' csv_to_fd -p {} {}'.format(
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
336 self.cls_name(), ' '.join(self.filenames)))
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
337 self.print_comment('Changes will be lost during the next update.')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
338 self.print_comment('Do not modify;'
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
339 ' modify the source .csv file instead.')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
340 self.print_header()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
341 print('')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
342 print('\n'.join(self.lines))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
343 self.print_header()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
344 self.print_comment('End of generated data.')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
345 self.print_header()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
346
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
347 def build_flags(self, flags):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
348 result = []
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
349 if 'V' in flags:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
350 result.append('AVP_FLAG_VENDOR')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
351 if 'M' in flags:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
352 result.append('AVP_FLAG_MANDATORY')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
353 return ' |'.join(result)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
354
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
355 def add(self, line):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
356 self.lines.append(line)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
357
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
358 def add_comment(self, comment):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
359 self.lines.append(self.format_comment(comment))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
360
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
361 def add_header(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
362 self.lines.append(self.format_header())
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
363
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
364 def format_comment(self, comment):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
365 return '\t/* {:<{width}} */'.format(comment, width=self.COMMENT_WIDTH)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
366
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
367 def format_header(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
368 return '\t/*={:=<{width}}=*/'.format('', width=self.COMMENT_WIDTH)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
369
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
370 def print_comment(self, comment):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
371 print(self.format_comment(comment))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
372
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
373 def print_header(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
374 print(self.format_header())
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
375
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
376
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
377 class JsonProcessor(Processor):
1494
ae76ea63ee12 csv_to_fd: improve comments and usage
Luke Mewburn <luke@mewburn.net>
parents: 1487
diff changeset
378 """Generate freeDiameter JSON object.
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
379 """
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
380
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
381 def __init__(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
382 self.avps = []
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
383
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
384 def next_file(self, filename):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
385 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
386
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
387 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
388 flags = collections.OrderedDict([
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
389 ('Must', self.build_flags(avp.must)),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
390 ('MustNot', self.build_flags(avp.mustnot)),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
391 ])
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
392 row = collections.OrderedDict([
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
393 ('Code', avp.code),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
394 ('Flags', flags),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
395 ('Name', avp.name),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
396 ('Type', avp.datatype),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
397 ('Vendor', avp.vendor),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
398 ])
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
399 self.avps.append(row)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
400
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
401 def comment(self, comment, filename, line_num):
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
402 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
403
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
404 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
405 doc = {"AVPs": self.avps}
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
406 print(json.dumps(doc, indent=2))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
407
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
408 def build_flags(self, flags):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
409 result = []
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
410 if 'V' in flags:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
411 result.append('V')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
412 if 'M' in flags:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
413 result.append('M')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
414 return ''.join(result)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
415
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
416
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
417 def avp_conflict(description, avp, conflict):
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
418 """Raise error for duplicate or conflicting AVPs.
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
419 """
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
420 if avp == conflict:
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
421 raise ValueError(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
422 '{} {} duplicated in'
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
423 ' file "{}" line {}'.format(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
424 avp.description(), description,
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
425 conflict.filename, conflict.line_num))
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
426 else:
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
427 raise ValueError(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
428 '{} {} conflicts with {}'
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
429 ' in file "{}" line {}'.format(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
430 avp.description(), description,
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
431 conflict.description(),
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
432 conflict.filename, conflict.line_num))
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
433
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
434
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
435 def main():
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
436 """Main application entry.
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
437 """
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
438
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
439 # Build dict of name: NameProcessor
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
440 processors = {
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
441 cls.cls_name(): cls
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
442 for cls in Processor.__subclasses__()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
443 }
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
444
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
445 # Build Processor name to desc
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
446 processor_help = '\n'.join(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
447 [' {:8} {}'.format(key, processors[key].cls_desc())
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
448 for key in sorted(processors)])
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
449
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
450 # Custom OptionParser with improved help
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
451 class MyParser(optparse.OptionParser):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
452 """Custom OptionParser without epilog formatting."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
453 def format_help(self, formatter=None):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
454 return """\
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
455 {}
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
456 Supported PROCESSOR options:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
457 {}
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
458 """.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
459 optparse.OptionParser.format_help(self, formatter),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
460 processor_help)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
462 # Parse options
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
463 parser = MyParser(
1494
ae76ea63ee12 csv_to_fd: improve comments and usage
Luke Mewburn <luke@mewburn.net>
parents: 1487
diff changeset
464 usage='%prog [-h] [-p PROCESSOR] FILE ...',
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
465 description="""\
1494
ae76ea63ee12 csv_to_fd: improve comments and usage
Luke Mewburn <luke@mewburn.net>
parents: 1487
diff changeset
466 Convert CSV files FILE ... containing RADIUS or Diameter AVP tables
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
467 into various formats using the specified processor PROCESSOR.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
468 """)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
469
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
470 parser.add_option(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
471 '-p', '--processor',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
472 default='noop',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
473 help='AVP processor. One of: {}. [%default]'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
474 ', '.join(processors.keys())))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
475 (opts, args) = parser.parse_args()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
476 if len(args) < 1:
1494
ae76ea63ee12 csv_to_fd: improve comments and usage
Luke Mewburn <luke@mewburn.net>
parents: 1487
diff changeset
477 parser.error('Incorrect number of arguments. Use -h for help.')
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
478
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
479 # Find processor
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
480 try:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
481 avpproc = processors[opts.processor]()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
482 except KeyError as e:
1494
ae76ea63ee12 csv_to_fd: improve comments and usage
Luke Mewburn <luke@mewburn.net>
parents: 1487
diff changeset
483 parser.error('Unknown processor "{}".'.format(opts.processor))
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
484
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
485 # dict of [vendor][code] : Avp
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
486 avp_codes = collections.defaultdict(dict)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
487
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
488 # dict of [vendor][name] : Avp
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
489 avp_names = collections.defaultdict(dict)
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
490
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
491 # Process files
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
492 for filename in args:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
493 avpproc.next_file(filename)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
494 with open(filename, 'r') as csvfile:
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
495 csvdata = csv.DictReader(csvfile, CSV_COLUMN_NAMES,
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
496 restkey='extra_cells')
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
497 standard = ''
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
498 vendor = 0
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
499 errors = []
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
500 for row in csvdata:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
501 try:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
502 if row['name'] in (None, '', 'Attribute Name'):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
503 continue
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
504 elif row['name'].startswith('#'):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
505 comment = row['name'][1:]
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
506 avpproc.comment(comment, filename, csvdata.line_num)
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
507 elif row['name'].startswith('@'):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
508 parameter = row['name'][1:]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
509 value = row['code']
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
510 if False:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
511 pass
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
512 elif parameter == 'standard':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
513 standard = value
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
514 elif parameter == 'vendor':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
515 vendor = int(value)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
516 else:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
517 raise ValueError('Unknown parameter "{}"'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
518 parameter))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
519 else:
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
520 avp = Avp(filename=filename, line_num=csvdata.line_num,
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
521 standard=standard, vendor=vendor,
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
522 **row)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
523 # Ensure AVP vendor/code not already defined
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
524 if avp.code in avp_codes[avp.vendor]:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
525 conflict = avp_codes[avp.vendor][avp.code]
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
526 avp_conflict('Code', avp, conflict)
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
527 avp_codes[avp.vendor][avp.code] = avp
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
528 # Ensure AVP vendor/name not already defined
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
529 if avp.name in avp_names[avp.vendor]:
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
530 conflict = avp_names[avp.vendor][avp.name]
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
531 avp_conflict('Name', avp, conflict)
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
532 avp_names[avp.vendor][avp.name] = avp
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
533 # Process AVP
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
534 avpproc.avp(avp)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
535 except ValueError as e:
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
536 errors.append('CSV file "{}" line {}: {}\n'.format(
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
537 filename, csvdata.line_num, e))
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
538 if errors:
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
539 sys.stderr.write(''.join(errors))
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
540 sys.exit(1)
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
541
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
542 # Generate result
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
543 avpproc.generate()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
544
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
545
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
546 if __name__ == '__main__':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
547 main()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
548
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
549 # vim: set et sw=4 sts=4 :
"Welcome to our mercurial repository"