Mercurial > hg > freeDiameter

annotate contrib/tools/csv_to_fd @ 1469:48fa8d70e6ad

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
csv_to_fd: validate flags and other improvements Validate the flags. Allow more columns; ignore any past column 8 "Must Not". Add basename of filenames to the regen command.
author Luke Mewburn <luke@mewburn.net>
date Wed, 18 Mar 2020 11:40:01 +1100
parents 1404de313b85
children c0aa1e66c12e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
1 #!/usr/bin/env python
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
2
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
3 """
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
4 Convert CSV files containing RADIUS or Diameter AVP tables
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
5 into various formats.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
6
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
7 Format of the CSV files is one of:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
8 - Row per 3GPP AVP tables:
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
9 Name, Code, Section, DataType, Must, May, ShouldNot, MustNot [, extra]
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
10 - Name:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
11 AVP Name. String, validated as ALPHA *(ALPHA / DIGIT / "-")
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
12 per RFC 6733 section 3.2.
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
13 - Code:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
14 AVP Code. Integer, 0..4294967295.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
15 - Section:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
16 Section in relevant standard. String.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
17 - DataType:
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
18 AVP Data Type. String, validated per basic and derived types in:
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
19 - RFC 6733 section 4.2
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
20 - RFC 6733 section 4.3
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
21 - RFC 7155 section 4.1
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
22 - Must, May, ShouldNot, MustNot:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
23 Flags, possibly comma or space separated: M, V
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
24
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
25 - Comment row. First cell:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
26 # Comment text Comment text
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
27 #= Header row of ====
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
28 # Blank line
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
29
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
30 - Parameter row:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
31 @Parameter,Value [, ...]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
32 Supported Parameter terms:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
33 standard Standard name. E.g. '3GPP TS 29.272', 'RFC 6733'.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
34 vendor Vendor number.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
35
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
36 """
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
37
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
38 from __future__ import print_function
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
39 from __future__ import with_statement
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
40
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
41 import abc
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
42 import csv
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
43 import collections
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
44 import json
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
45 import re
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
46 import optparse
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
47 import os
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
48 import sys
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
49
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
50 CSV_COLUMN_NAMES = [
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
51 'name',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
52 'code',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
53 'section',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
54 'datatype',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
55 'must',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
56 'may',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
57 'shouldnot',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
58 'mustnot',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
59 ]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
60
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
61 DERIVED_TO_BASE = {
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
62 'Address': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
63 'Time': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
64 'UTF8String': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
65 'DiameterIdentity': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
66 'DiameterURI': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
67 'Enumerated': 'Integer32', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
68 'IPFilterRule': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
69 'QoSFilterRule': 'OctetString', # RFC 7155 section 4.1.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
70 }
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
71
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
72 # See https://www.iana.org/assignments/enterprise-numbers/enterprise-numbers
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
73 VENDOR_TO_NAME = {
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
74 0: '',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
75 193: 'Ericsson',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
76 8164: 'Starent',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
77 10415: '3GPP',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
78 }
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
79
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
80
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
81 class Avp(object):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
82 """Store an AVP row."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
83
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
84 # Regex to validate avp-name per RFC 6733 section 3.2,
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
85 # with changes:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
86 # - Allow avp-name to start with numbers (for 3GPP)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
87 # - Allow '.' in avp-name, for existing dict_dcca_3gpp usage.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
88 # TODO: if starts with digit, ensure contains a letter somewhere?
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
89 _name_re = re.compile(r'^[a-zA-Z0-9][a-zA-Z0-9-\.]*$')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
90
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
91 # Regex to validate flags: M, P, V, comma, space
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
92 _flags_re = re.compile(r'^[MPV, ]*$')
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
93
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
94 __slots__ = CSV_COLUMN_NAMES + [
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
95 'filename', 'linenum', 'standard', 'vendor', ]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
96
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
97 def __init__(self, name, code, section, datatype,
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
98 must, may, shouldnot, mustnot, extra_cells,
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
99 filename='', linenum=0, standard='', vendor=0):
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
100 # Members from CSV row
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
101 self.name = name
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
102 self.code = int(code)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
103 self.section = section
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
104 self.datatype = datatype
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
105 self.must = must
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
106 self.may = may
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
107 self.shouldnot = shouldnot
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
108 self.mustnot = mustnot
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
109 # Members from file state
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
110 self.filename = filename
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
111 self.linenum = linenum
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
112 self.standard = standard
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
113 self.vendor = vendor
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
114 # Validate CSV fields
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
115 if not self._name_re.match(self.name):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
116 raise ValueError('Invalid AVP name "{}"'.format(self.name))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
117 if (self.code < 0 or self.code > 4294967295):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
118 raise ValueError('Invalid AVP code {}'.format(self.code))
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
119 if (self.datatype not in (
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
120 'OctetString', 'Integer32', 'Integer64', 'Unsigned32',
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
121 'Unsigned64', 'Float32', 'Float64', 'Grouped')
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
122 and self.datatype not in DERIVED_TO_BASE):
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
123 raise ValueError('Invalid AVP data type "{}"'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
124 self.datatype))
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
125 for val, desc in [
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
126 (self.must, 'Must'),
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
127 (self.may, 'May'),
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
128 (self.shouldnot, 'Should Not'),
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
129 (self.mustnot, 'Must Not'),
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
130 ]:
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
131 if not self._flags_re.match(val):
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
132 raise ValueError('Invalid AVP Flags {} "{}"'.format(desc, val))
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
133
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
134 @property
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
135 def __dict__(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
136 return {s: getattr(self, s) for s in self.__slots__}
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
137
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
138
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
139 class Processor(object):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
140 """Interface for processor of Avp"""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
141
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
142 __metaclass__ = abc.ABCMeta
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
143
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
144 @classmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
145 def cls_name(cls):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
146 """Return the name, lower-case, without "processor" suffix."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
147 suffix = 'processor'
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
148 name = cls.__name__.lower()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
149 if name.endswith(suffix):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
150 return name[:-len(suffix)]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
151 return name
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
152
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
153 @classmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
154 def cls_desc(cls):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
155 """Return the first line of the docstring."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
156 if cls.__doc__ is None:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
157 return ""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
158 return cls.__doc__.split('\n')[0]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
159
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
160 @abc.abstractmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
161 def next_file(self, filename):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
162 """Called when a file is opened."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
163 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
164
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
165 @abc.abstractmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
166 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
167 """Process a validated Avp."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
168 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
169
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
170 @abc.abstractmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
171 def comment(self, comment, filename, linenum):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
172 """Process a comment row:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
173 #comment,
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
174 """
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
175 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
176
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
177 @abc.abstractmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
178 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
179 """Invoked after all rows processed."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
180 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
181
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
182
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
183 class DebugProcessor(Processor):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
184 """Display the CSV parsing"""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
185
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
186 def next_file(self, filename):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
187 print('File: {}'.format(filename))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
188
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
189 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
190 avpdict = vars(avp)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
191 print('AVP: {name}, {code}, {datatype}'.format(**avpdict))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
192
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
193 def comment(self, comment, filename, linenum):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
194 print('Comment: {}'.format(comment))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
195
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
196 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
197 print('Generate')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
198
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
199
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
200 class NoopProcessor(Processor):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
201 """Validate the CSV; no other output"""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
202
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
203 def next_file(self, filename):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
204 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
205
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
206 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
207 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
208
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
209 def comment(self, comment, filename, linenum):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
210 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
211
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
212 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
213 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
214
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
215
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
216 class FdcProcessor(Processor):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
217 """Generate freeDiameter C code
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
218
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
219 Comment cells are parsed as:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
220 # text comment /* text comment */
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
221 #= /*==============*/
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
222 # [blank line]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
223 """
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
224
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
225 COMMENT_WIDTH = 64
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
226
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
227 def __init__(self):
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
228 self.filenames = []
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
229 self.lines = []
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
230
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
231 def next_file(self, filename):
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
232 self.filenames.append(os.path.basename(filename))
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
233
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
234 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
235 comment = '{name}, {datatype}, code {code}'.format(**vars(avp))
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
236 if avp.section != '':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
237 comment += ', section {}'.format(avp.section)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
238 self.add_comment(comment)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
239 self.add('\t{')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
240 self.add('\t\tstruct dict_avp_data data = {')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
241 # TODO: remove comments?
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
242 self.add('\t\t\t{},\t/* Code */'.format(avp.code))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
243 self.add('\t\t\t{},\t/* Vendor */'.format(avp.vendor))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
244 self.add('\t\t\t\"{}\",\t/* Name */'.format(avp.name))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
245 self.add('\t\t\t{},\t/* Fixed flags */'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
246 self.build_flags(', '.join([avp.must, avp.mustnot]))))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
247 self.add('\t\t\t{},\t/* Fixed flag values */'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
248 self.build_flags(avp.must)))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
249 # TODO: add trailing comma?
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
250 self.add('\t\t\tAVP_TYPE_{}\t/* base type of data */'.format(
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
251 DERIVED_TO_BASE.get(avp.datatype, avp.datatype).upper()))
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
252 self.add('\t\t};')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
253 avp_type = 'NULL'
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
254 if avp.datatype == 'Enumerated':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
255 self.add('\t\tstruct dict_object\t*type;')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
256 vendor_prefix = ''
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
257 if avp.vendor != 0:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
258 vendor_prefix = '{}/'.format(VENDOR_TO_NAME[avp.vendor])
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
259 self.add(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
260 '\t\tstruct dict_type_data\t tdata = {{ AVP_TYPE_INTEGER32, '
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
261 '"Enumerated({prefix}{name})", NULL, NULL, NULL }};'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
262 prefix=vendor_prefix, name=avp.name))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
263 # XXX: add enumerated values
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
264 self.add('\t\tCHECK_dict_new(DICT_TYPE, &tdata, NULL, &type);')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
265 avp_type = "type"
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
266 elif avp.datatype in DERIVED_TO_BASE:
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
267 avp_type = '{}_type'.format(avp.datatype)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
268 self.add('\t\tCHECK_dict_new(DICT_AVP, &data, {}, NULL);'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
269 avp_type))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
270 # TODO: remove ; on scope brace
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
271 self.add('\t};')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
272 self.add('')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
273
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
274 def comment(self, comment, filename, linenum):
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
275 if comment == '':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
276 self.add('')
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
277 elif comment == '=':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
278 self.add_header()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
279 elif comment.startswith(' '):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
280 self.add_comment(comment[1:])
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
281 else:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
282 raise ValueError('Unsupported comment "{}"'.format(comment))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
283
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
284 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
285 self.print_header()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
286 self.print_comment('Start of generated data.')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
287 self.print_comment('')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
288 self.print_comment('The following is created automatically with:')
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
289 self.print_comment(' csv_to_fd -p {} {}'.format(
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
290 self.cls_name(), ' '.join(self.filenames)))
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
291 self.print_comment('Changes will be lost during the next update.')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
292 self.print_comment('Do not modify;'
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
293 ' modify the source .csv file instead.')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
294 self.print_header()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
295 print('')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
296 print('\n'.join(self.lines))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
297 self.print_header()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
298 self.print_comment('End of generated data.')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
299 self.print_header()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
300
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
301 def build_flags(self, flags):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
302 result = []
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
303 if 'V' in flags:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
304 result.append('AVP_FLAG_VENDOR')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
305 if 'M' in flags:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
306 result.append('AVP_FLAG_MANDATORY')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
307 return ' |'.join(result)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
308
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
309 def add(self, line):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
310 self.lines.append(line)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
311
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
312 def add_comment(self, comment):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
313 self.lines.append(self.format_comment(comment))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
314
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
315 def add_header(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
316 self.lines.append(self.format_header())
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
317
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
318 def format_comment(self, comment):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
319 return '\t/* {:<{width}} */'.format(comment, width=self.COMMENT_WIDTH)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
320
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
321 def format_header(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
322 return '\t/*={:=<{width}}=*/'.format('', width=self.COMMENT_WIDTH)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
323
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
324 def print_comment(self, comment):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
325 print(self.format_comment(comment))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
326
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
327 def print_header(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
328 print(self.format_header())
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
329
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
330
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
331 class JsonProcessor(Processor):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
332 """Generate freeDiameter JSON object
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
333 """
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
334
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
335 def __init__(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
336 self.avps = []
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
337
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
338 def next_file(self, filename):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
339 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
340
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
341 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
342 flags = collections.OrderedDict([
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
343 ('Must', self.build_flags(avp.must)),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
344 ('MustNot', self.build_flags(avp.mustnot)),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
345 ])
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
346 row = collections.OrderedDict([
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
347 ('Code', avp.code),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
348 ('Flags', flags),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
349 ('Name', avp.name),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
350 ('Type', avp.datatype),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
351 ('Vendor', avp.vendor),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
352 ])
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
353 self.avps.append(row)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
354
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
355 def comment(self, comment, filename, linenum):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
356 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
357
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
358 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
359 doc = {"AVPs": self.avps}
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
360 print(json.dumps(doc, indent=2))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
361
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
362 def build_flags(self, flags):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
363 result = []
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
364 if 'V' in flags:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
365 result.append('V')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
366 if 'M' in flags:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
367 result.append('M')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
368 return ''.join(result)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
369
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
370
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
371 def main():
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
372
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
373 # Build dict of name: NameProcessor
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
374 processors = {
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
375 cls.cls_name(): cls
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
376 for cls in Processor.__subclasses__()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
377 }
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
378
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
379 # Build Processor name to desc
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
380 processor_help = '\n'.join(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
381 [' {:8} {}'.format(key, processors[key].cls_desc())
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
382 for key in sorted(processors)])
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
383
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
384 # Custom OptionParser with improved help
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
385 class MyParser(optparse.OptionParser):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
386 """Custom OptionParser without epilog formatting."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
387 def format_help(self, formatter=None):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
388 return """\
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
389 {}
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
390 Supported PROCESSOR options:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
391 {}
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
392 """.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
393 optparse.OptionParser.format_help(self, formatter),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
394 processor_help)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
395
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
396 # Parse options
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
397 parser = MyParser(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
398 description="""\
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
399 Convert CSV files containing RADIUS or Diameter AVP tables
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
400 into various formats using the specified processor PROCESSOR.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
401 """)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
402
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
403 parser.add_option(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
404 '-p', '--processor',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
405 default='noop',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
406 help='AVP processor. One of: {}. [%default]'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
407 ', '.join(processors.keys())))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
408 (opts, args) = parser.parse_args()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
409 if len(args) < 1:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
410 parser.error('Incorrect number of arguments')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
411
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
412 # Find processor
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
413 try:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
414 avpproc = processors[opts.processor]()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
415 except KeyError as e:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
416 parser.error('Unknown processor "{}"'.format(opts.processor))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
417
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
418 # dict of [vendor][code] : Avp
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
419 avp_codes = collections.defaultdict(dict)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
420
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
421 # Process files
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
422 for filename in args:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
423 avpproc.next_file(filename)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
424 with open(filename, 'r') as csvfile:
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
425 csvdata = csv.DictReader(csvfile, CSV_COLUMN_NAMES,
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
426 restkey='extra_cells')
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
427 linenum = 0
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
428 standard = ''
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
429 vendor = 0
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
430 for row in csvdata:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
431 linenum += 1
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
432 try:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
433 if row['name'] in (None, '', 'Attribute Name'):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
434 continue
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
435 elif row['name'].startswith('#'):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
436 comment = row['name'][1:]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
437 avpproc.comment(comment, filename, linenum)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
438 elif row['name'].startswith('@'):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
439 parameter = row['name'][1:]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
440 value = row['code']
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
441 if False:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
442 pass
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
443 elif parameter == 'standard':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
444 standard = value
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
445 elif parameter == 'vendor':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
446 vendor = int(value)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
447 else:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
448 raise ValueError('Unknown parameter "{}"'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
449 parameter))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
450 else:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
451 avp = Avp(filename=filename, linenum=linenum,
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
452 standard=standard, vendor=vendor,
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
453 **row)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
454 # Ensure AVP vendor/code not already defined
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
455 if avp.code in avp_codes[avp.vendor]:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
456 conflict = avp_codes[avp.vendor][avp.code]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
457 raise ValueError(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
458 'AVP vendor {} code {} already present'
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
459 ' in file "{}" line {}'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
460 avp.vendor, avp.code,
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
461 conflict.filename, conflict.linenum))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
462 avp_codes[avp.vendor][avp.code] = avp
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
463 # Process AVP
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
464 avpproc.avp(avp)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
465 except ValueError as e:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
466 sys.stderr.write('CSV file "{}" line {}: {}: {}\n'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
467 filename, linenum, e.__class__.__name__, e))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
468 sys.exit(1)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
469
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
470 # Generate result
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
471 avpproc.generate()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
472
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
473
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
474 if __name__ == '__main__':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
475 main()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
476
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
477 # vim: set et sw=4 sts=4 :
"Welcome to our mercurial repository"