Mercurial > hg > freeDiameter
annotate contrib/tools/csv_to_fd @ 1479:c0aa1e66c12e
csv_to_fd: improve validation
* Validate flags:
- Ensure M and V can't occur more than once.
- Ensure absence of V for vendor 0.
- Ensure presence of V for vendor not 0.
* Ensure AVP name is unique per vendor.
* Add AVP name to most errors.
* Use csv.DictReader.line_num for more accurate line numbers.
TODO: enforce M and V must be present once and only once.
Some AVPs currently fail that rule, including:
- RFC 4005 / RFC 7155 QoS-Filter-Rule
- 3GPP TS 29.212 PDN-Connection-ID
- 3GPP TS 29.212 PS-to-CS-Session-Continuity
author | Luke Mewburn <luke@mewburn.net> |
---|---|
date | Thu, 26 Mar 2020 10:28:16 +1100 |
parents | 48fa8d70e6ad |
children | 5c2d061a8c8e |
rev | line source |
---|---|
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
1 #!/usr/bin/env python |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
2 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
3 """ |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
4 Convert CSV files containing RADIUS or Diameter AVP tables |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
5 into various formats. |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
6 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
7 Format of the CSV files is one of: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
8 - Row per 3GPP AVP tables: |
1469
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
9 Name, Code, Section, DataType, Must, May, ShouldNot, MustNot [, extra] |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
10 - Name: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
11 AVP Name. String, validated as ALPHA *(ALPHA / DIGIT / "-") |
1463
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
12 per RFC 6733 section 3.2. |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
13 - Code: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
14 AVP Code. Integer, 0..4294967295. |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
15 - Section: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
16 Section in relevant standard. String. |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
17 - DataType: |
1463
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
18 AVP Data Type. String, validated per basic and derived types in: |
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
19 - RFC 6733 section 4.2 |
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
20 - RFC 6733 section 4.3 |
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
21 - RFC 7155 section 4.1 |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
22 - Must, May, ShouldNot, MustNot: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
23 Flags, possibly comma or space separated: M, V |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
24 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
25 - Comment row. First cell: |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
26 # Comment text 'Comment text' |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
27 #= '/*========*/' |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
28 # Blank line |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
29 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
30 - Parameter row: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
31 @Parameter,Value [, ...] |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
32 Supported Parameter terms: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
33 standard Standard name. E.g. '3GPP TS 29.272', 'RFC 6733'. |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
34 vendor Vendor number. |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
35 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
36 """ |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
37 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
38 from __future__ import print_function |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
39 from __future__ import with_statement |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
40 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
41 import abc |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
42 import csv |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
43 import collections |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
44 import json |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
45 import re |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
46 import optparse |
1469
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
47 import os |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
48 import sys |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
49 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
50 CSV_COLUMN_NAMES = [ |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
51 'name', |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
52 'code', |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
53 'section', |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
54 'datatype', |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
55 'must', |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
56 'may', |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
57 'shouldnot', |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
58 'mustnot', |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
59 ] |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
60 |
1463
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
61 DERIVED_TO_BASE = { |
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
62 'Address': 'OctetString', # RFC 6733 section 4.3.1 |
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
63 'Time': 'OctetString', # RFC 6733 section 4.3.1 |
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
64 'UTF8String': 'OctetString', # RFC 6733 section 4.3.1 |
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
65 'DiameterIdentity': 'OctetString', # RFC 6733 section 4.3.1 |
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
66 'DiameterURI': 'OctetString', # RFC 6733 section 4.3.1 |
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
67 'Enumerated': 'Integer32', # RFC 6733 section 4.3.1 |
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
68 'IPFilterRule': 'OctetString', # RFC 6733 section 4.3.1 |
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
69 'QoSFilterRule': 'OctetString', # RFC 7155 section 4.1.1 |
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
70 } |
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
71 |
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
72 # See https://www.iana.org/assignments/enterprise-numbers/enterprise-numbers |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
73 VENDOR_TO_NAME = { |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
74 0: '', |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
75 193: 'Ericsson', |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
76 8164: 'Starent', |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
77 10415: '3GPP', |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
78 } |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
79 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
80 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
81 class Avp(object): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
82 """Store an AVP row.""" |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
83 |
1463
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
84 # Regex to validate avp-name per RFC 6733 section 3.2, |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
85 # with changes: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
86 # - Allow avp-name to start with numbers (for 3GPP) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
87 # - Allow '.' in avp-name, for existing dict_dcca_3gpp usage. |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
88 # TODO: if starts with digit, ensure contains a letter somewhere? |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
89 _name_re = re.compile(r'^[a-zA-Z0-9][a-zA-Z0-9-\.]*$') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
90 |
1469
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
91 # Regex to validate flags: M, P, V, comma, space |
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
92 _flags_re = re.compile(r'^[MPV, ]*$') |
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
93 |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
94 __slots__ = CSV_COLUMN_NAMES + [ |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
95 'filename', 'line_num', 'standard', 'vendor', ] |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
96 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
97 def __init__(self, name, code, section, datatype, |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
98 must, may, shouldnot, mustnot, extra_cells=[], |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
99 filename='', line_num=0, standard='', vendor=0): |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
100 # Members from CSV row |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
101 self.name = name |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
102 self.code = int(code) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
103 self.section = section |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
104 self.datatype = datatype |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
105 self.must = must |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
106 self.may = may |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
107 self.shouldnot = shouldnot |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
108 self.mustnot = mustnot |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
109 # Members from file state |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
110 self.filename = filename |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
111 self.line_num = line_num |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
112 self.standard = standard |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
113 self.vendor = vendor |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
114 # Validate CSV fields |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
115 if not self._name_re.match(self.name): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
116 raise ValueError('Invalid AVP name "{}"'.format(self.name)) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
117 if (self.code < 0 or self.code > 4294967295): |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
118 raise ValueError('AVP "{}" invalid code {}'.format( |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
119 self.name, self.code)) |
1463
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
120 if (self.datatype not in ( |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
121 'OctetString', 'Integer32', 'Integer64', 'Unsigned32', |
1463
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
122 'Unsigned64', 'Float32', 'Float64', 'Grouped') |
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
123 and self.datatype not in DERIVED_TO_BASE): |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
124 raise ValueError('AVP "{}" invalid data type "{}"'.format( |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
125 self.name, self.datatype)) |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
126 # Validate flags |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
127 flags = collections.Counter() |
1469
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
128 for val, desc in [ |
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
129 (self.must, 'Must'), |
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
130 (self.may, 'May'), |
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
131 (self.shouldnot, 'Should Not'), |
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
132 (self.mustnot, 'Must Not'), |
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
133 ]: |
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
134 if not self._flags_re.match(val): |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
135 raise ValueError('AVP "{}" invalid {} Flags "{}"'.format( |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
136 self.name, desc, val)) |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
137 flags.update(val) |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
138 # Check occurrence of M,V in Must,May,ShouldNot,MustNot |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
139 for flag in 'MV': |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
140 # TODO: can AVP flags not appear at all? |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
141 # if flags[flag] == 0: |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
142 # raise ValueError('AVP "{}" Flag "{}" not set'.format( |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
143 # self.name, flag)) |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
144 if flags[flag] > 1: |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
145 raise ValueError('AVP "{}" Flag "{}" set {} times'.format( |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
146 self.name, flag, flags[flag])) |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
147 # Compare V presence against vendor |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
148 if 'V' in self.must: |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
149 if self.vendor == 0: |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
150 raise ValueError('AVP "{}" Flag "V" set for vendor 0'.format( |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
151 self.name)) |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
152 else: |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
153 if self.vendor != 0: |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
154 raise ValueError( |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
155 'AVP "{}" Flag "V" not set for vendor {}'.format( |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
156 self.name, self.vendor)) |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
157 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
158 @property |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
159 def __dict__(self): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
160 return {s: getattr(self, s) for s in self.__slots__} |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
161 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
162 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
163 class Processor(object): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
164 """Interface for processor of Avp""" |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
165 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
166 __metaclass__ = abc.ABCMeta |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
167 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
168 @classmethod |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
169 def cls_name(cls): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
170 """Return the name, lower-case, without "processor" suffix.""" |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
171 suffix = 'processor' |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
172 name = cls.__name__.lower() |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
173 if name.endswith(suffix): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
174 return name[:-len(suffix)] |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
175 return name |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
176 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
177 @classmethod |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
178 def cls_desc(cls): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
179 """Return the first line of the docstring.""" |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
180 if cls.__doc__ is None: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
181 return "" |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
182 return cls.__doc__.split('\n')[0] |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
183 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
184 @abc.abstractmethod |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
185 def next_file(self, filename): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
186 """Called when a file is opened.""" |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
187 pass |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
188 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
189 @abc.abstractmethod |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
190 def avp(self, avp): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
191 """Process a validated Avp.""" |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
192 pass |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
193 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
194 @abc.abstractmethod |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
195 def comment(self, comment, filename, line_num): |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
196 """Process a comment row: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
197 #comment, |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
198 """ |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
199 pass |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
200 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
201 @abc.abstractmethod |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
202 def generate(self): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
203 """Invoked after all rows processed.""" |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
204 pass |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
205 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
206 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
207 class DebugProcessor(Processor): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
208 """Display the CSV parsing""" |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
209 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
210 def next_file(self, filename): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
211 print('File: {}'.format(filename)) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
212 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
213 def avp(self, avp): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
214 avpdict = vars(avp) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
215 print('AVP: {name}, {code}, {datatype}'.format(**avpdict)) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
216 |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
217 def comment(self, comment, filename, line_num): |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
218 print('Comment: {}'.format(comment)) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
219 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
220 def generate(self): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
221 print('Generate') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
222 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
223 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
224 class NoopProcessor(Processor): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
225 """Validate the CSV; no other output""" |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
226 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
227 def next_file(self, filename): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
228 pass |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
229 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
230 def avp(self, avp): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
231 pass |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
232 |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
233 def comment(self, comment, filename, line_num): |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
234 pass |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
235 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
236 def generate(self): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
237 pass |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
238 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
239 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
240 class FdcProcessor(Processor): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
241 """Generate freeDiameter C code |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
242 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
243 Comment cells are parsed as: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
244 # text comment /* text comment */ |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
245 #= /*==============*/ |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
246 # [blank line] |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
247 """ |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
248 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
249 COMMENT_WIDTH = 64 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
250 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
251 def __init__(self): |
1469
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
252 self.filenames = [] |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
253 self.lines = [] |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
254 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
255 def next_file(self, filename): |
1469
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
256 self.filenames.append(os.path.basename(filename)) |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
257 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
258 def avp(self, avp): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
259 comment = '{name}, {datatype}, code {code}'.format(**vars(avp)) |
1464
1404de313b85
csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents:
1463
diff
changeset
|
260 if avp.section != '': |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
261 comment += ', section {}'.format(avp.section) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
262 self.add_comment(comment) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
263 self.add('\t{') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
264 self.add('\t\tstruct dict_avp_data data = {') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
265 # TODO: remove comments? |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
266 self.add('\t\t\t{},\t/* Code */'.format(avp.code)) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
267 self.add('\t\t\t{},\t/* Vendor */'.format(avp.vendor)) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
268 self.add('\t\t\t\"{}\",\t/* Name */'.format(avp.name)) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
269 self.add('\t\t\t{},\t/* Fixed flags */'.format( |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
270 self.build_flags(', '.join([avp.must, avp.mustnot])))) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
271 self.add('\t\t\t{},\t/* Fixed flag values */'.format( |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
272 self.build_flags(avp.must))) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
273 # TODO: add trailing comma? |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
274 self.add('\t\t\tAVP_TYPE_{}\t/* base type of data */'.format( |
1463
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
275 DERIVED_TO_BASE.get(avp.datatype, avp.datatype).upper())) |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
276 self.add('\t\t};') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
277 avp_type = 'NULL' |
1464
1404de313b85
csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents:
1463
diff
changeset
|
278 if avp.datatype == 'Enumerated': |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
279 self.add('\t\tstruct dict_object\t*type;') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
280 vendor_prefix = '' |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
281 if avp.vendor != 0: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
282 vendor_prefix = '{}/'.format(VENDOR_TO_NAME[avp.vendor]) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
283 self.add( |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
284 '\t\tstruct dict_type_data\t tdata = {{ AVP_TYPE_INTEGER32, ' |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
285 '"Enumerated({prefix}{name})", NULL, NULL, NULL }};'.format( |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
286 prefix=vendor_prefix, name=avp.name)) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
287 # XXX: add enumerated values |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
288 self.add('\t\tCHECK_dict_new(DICT_TYPE, &tdata, NULL, &type);') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
289 avp_type = "type" |
1463
8f6c77f24b1a
csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents:
1461
diff
changeset
|
290 elif avp.datatype in DERIVED_TO_BASE: |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
291 avp_type = '{}_type'.format(avp.datatype) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
292 self.add('\t\tCHECK_dict_new(DICT_AVP, &data, {}, NULL);'.format( |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
293 avp_type)) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
294 # TODO: remove ; on scope brace |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
295 self.add('\t};') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
296 self.add('') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
297 |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
298 def comment(self, comment, filename, line_num): |
1464
1404de313b85
csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents:
1463
diff
changeset
|
299 if comment == '': |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
300 self.add('') |
1464
1404de313b85
csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents:
1463
diff
changeset
|
301 elif comment == '=': |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
302 self.add_header() |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
303 elif comment.startswith(' '): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
304 self.add_comment(comment[1:]) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
305 else: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
306 raise ValueError('Unsupported comment "{}"'.format(comment)) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
307 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
308 def generate(self): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
309 self.print_header() |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
310 self.print_comment('Start of generated data.') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
311 self.print_comment('') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
312 self.print_comment('The following is created automatically with:') |
1469
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
313 self.print_comment(' csv_to_fd -p {} {}'.format( |
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
314 self.cls_name(), ' '.join(self.filenames))) |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
315 self.print_comment('Changes will be lost during the next update.') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
316 self.print_comment('Do not modify;' |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
317 ' modify the source .csv file instead.') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
318 self.print_header() |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
319 print('') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
320 print('\n'.join(self.lines)) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
321 self.print_header() |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
322 self.print_comment('End of generated data.') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
323 self.print_header() |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
324 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
325 def build_flags(self, flags): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
326 result = [] |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
327 if 'V' in flags: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
328 result.append('AVP_FLAG_VENDOR') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
329 if 'M' in flags: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
330 result.append('AVP_FLAG_MANDATORY') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
331 return ' |'.join(result) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
332 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
333 def add(self, line): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
334 self.lines.append(line) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
335 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
336 def add_comment(self, comment): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
337 self.lines.append(self.format_comment(comment)) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
338 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
339 def add_header(self): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
340 self.lines.append(self.format_header()) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
341 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
342 def format_comment(self, comment): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
343 return '\t/* {:<{width}} */'.format(comment, width=self.COMMENT_WIDTH) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
344 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
345 def format_header(self): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
346 return '\t/*={:=<{width}}=*/'.format('', width=self.COMMENT_WIDTH) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
347 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
348 def print_comment(self, comment): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
349 print(self.format_comment(comment)) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
350 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
351 def print_header(self): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
352 print(self.format_header()) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
353 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
354 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
355 class JsonProcessor(Processor): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
356 """Generate freeDiameter JSON object |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
357 """ |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
358 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
359 def __init__(self): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
360 self.avps = [] |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
361 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
362 def next_file(self, filename): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
363 pass |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
364 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
365 def avp(self, avp): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
366 flags = collections.OrderedDict([ |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
367 ('Must', self.build_flags(avp.must)), |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
368 ('MustNot', self.build_flags(avp.mustnot)), |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
369 ]) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
370 row = collections.OrderedDict([ |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
371 ('Code', avp.code), |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
372 ('Flags', flags), |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
373 ('Name', avp.name), |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
374 ('Type', avp.datatype), |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
375 ('Vendor', avp.vendor), |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
376 ]) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
377 self.avps.append(row) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
378 |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
379 def comment(self, comment, filename, line_num): |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
380 pass |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
381 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
382 def generate(self): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
383 doc = {"AVPs": self.avps} |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
384 print(json.dumps(doc, indent=2)) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
385 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
386 def build_flags(self, flags): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
387 result = [] |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
388 if 'V' in flags: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
389 result.append('V') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
390 if 'M' in flags: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
391 result.append('M') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
392 return ''.join(result) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
393 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
394 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
395 def main(): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
396 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
397 # Build dict of name: NameProcessor |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
398 processors = { |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
399 cls.cls_name(): cls |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
400 for cls in Processor.__subclasses__() |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
401 } |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
402 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
403 # Build Processor name to desc |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
404 processor_help = '\n'.join( |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
405 [' {:8} {}'.format(key, processors[key].cls_desc()) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
406 for key in sorted(processors)]) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
407 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
408 # Custom OptionParser with improved help |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
409 class MyParser(optparse.OptionParser): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
410 """Custom OptionParser without epilog formatting.""" |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
411 def format_help(self, formatter=None): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
412 return """\ |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
413 {} |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
414 Supported PROCESSOR options: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
415 {} |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
416 """.format( |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
417 optparse.OptionParser.format_help(self, formatter), |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
418 processor_help) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
419 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
420 # Parse options |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
421 parser = MyParser( |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
422 description="""\ |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
423 Convert CSV files containing RADIUS or Diameter AVP tables |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
424 into various formats using the specified processor PROCESSOR. |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
425 """) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
426 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
427 parser.add_option( |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
428 '-p', '--processor', |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
429 default='noop', |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
430 help='AVP processor. One of: {}. [%default]'.format( |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
431 ', '.join(processors.keys()))) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
432 (opts, args) = parser.parse_args() |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
433 if len(args) < 1: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
434 parser.error('Incorrect number of arguments') |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
435 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
436 # Find processor |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
437 try: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
438 avpproc = processors[opts.processor]() |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
439 except KeyError as e: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
440 parser.error('Unknown processor "{}"'.format(opts.processor)) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
441 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
442 # dict of [vendor][code] : Avp |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
443 avp_codes = collections.defaultdict(dict) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
444 |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
445 # dict of [vendor][name] : Avp |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
446 avp_names = collections.defaultdict(dict) |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
447 |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
448 # Process files |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
449 for filename in args: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
450 avpproc.next_file(filename) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
451 with open(filename, 'r') as csvfile: |
1469
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
452 csvdata = csv.DictReader(csvfile, CSV_COLUMN_NAMES, |
48fa8d70e6ad
csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents:
1464
diff
changeset
|
453 restkey='extra_cells') |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
454 standard = '' |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
455 vendor = 0 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
456 for row in csvdata: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
457 try: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
458 if row['name'] in (None, '', 'Attribute Name'): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
459 continue |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
460 elif row['name'].startswith('#'): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
461 comment = row['name'][1:] |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
462 avpproc.comment(comment, filename, csvdata.line_num) |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
463 elif row['name'].startswith('@'): |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
464 parameter = row['name'][1:] |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
465 value = row['code'] |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
466 if False: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
467 pass |
1464
1404de313b85
csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents:
1463
diff
changeset
|
468 elif parameter == 'standard': |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
469 standard = value |
1464
1404de313b85
csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents:
1463
diff
changeset
|
470 elif parameter == 'vendor': |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
471 vendor = int(value) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
472 else: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
473 raise ValueError('Unknown parameter "{}"'.format( |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
474 parameter)) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
475 else: |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
476 avp = Avp(filename=filename, line_num=csvdata.line_num, |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
477 standard=standard, vendor=vendor, |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
478 **row) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
479 # Ensure AVP vendor/code not already defined |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
480 if avp.code in avp_codes[avp.vendor]: |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
481 conflict = avp_codes[avp.vendor][avp.code] |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
482 raise ValueError( |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
483 'AVP vendor {} code {} already present' |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
484 ' in file "{}" line {}'.format( |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
485 avp.vendor, avp.code, |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
486 conflict.filename, conflict.line_num)) |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
487 avp_codes[avp.vendor][avp.code] = avp |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
488 # Ensure AVP vendor/name not already defined |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
489 if avp.name in avp_names[avp.vendor]: |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
490 conflict = avp_names[avp.vendor][avp.name] |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
491 raise ValueError( |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
492 'AVP vendor {} name "{}" already present' |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
493 ' in file "{}" line {}'.format( |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
494 avp.vendor, avp.name, |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
495 conflict.filename, conflict.line_num)) |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
496 avp_names[avp.vendor][avp.name] = avp |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
497 # Process AVP |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
498 avpproc.avp(avp) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
499 except ValueError as e: |
1479
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
500 sys.stderr.write('CSV file "{}" line {}: {}\n'.format( |
c0aa1e66c12e
csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents:
1469
diff
changeset
|
501 filename, csvdata.line_num, e)) |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
502 sys.exit(1) |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
503 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
504 # Generate result |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
505 avpproc.generate() |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
506 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
507 |
1464
1404de313b85
csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents:
1463
diff
changeset
|
508 if __name__ == '__main__': |
1461
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
509 main() |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
510 |
a86eb3375b95
add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff
changeset
|
511 # vim: set et sw=4 sts=4 : |