Mercurial > hg > freeDiameter

annotate contrib/tools/csv_to_fd @ 1487:5c2d061a8c8e

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
csv_to_fd: improve validation When an AVP Code or Name is duplicate, differentiate between a duplicate entry (AVPs are equivalent) versus an actual mismatch. Consistent AVP formatting in errors. Collect all errors and display at end of each file parse, instead of exiting on first error.
author Luke Mewburn <luke@mewburn.net>
date Fri, 27 Mar 2020 15:32:27 +1100
parents c0aa1e66c12e
children ae76ea63ee12
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
1 #!/usr/bin/env python
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
2
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
3 """
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
4 Convert CSV files containing RADIUS or Diameter AVP tables
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
5 into various formats.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
6
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
7 Format of the CSV files is one of:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
8 - Row per 3GPP AVP tables:
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
9 Name, Code, Section, DataType, Must, May, ShouldNot, MustNot [, extra]
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
10 - Name:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
11 AVP Name. String, validated as ALPHA *(ALPHA / DIGIT / "-")
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
12 per RFC 6733 section 3.2.
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
13 - Code:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
14 AVP Code. Integer, 0..4294967295.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
15 - Section:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
16 Section in relevant standard. String.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
17 - DataType:
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
18 AVP Data Type. String, validated per basic and derived types in:
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
19 - RFC 6733 section 4.2
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
20 - RFC 6733 section 4.3
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
21 - RFC 7155 section 4.1
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
22 - Must, May, ShouldNot, MustNot:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
23 Flags, possibly comma or space separated: M, V
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
24
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
25 - Comment row. First cell:
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
26 # Comment text 'Comment text'
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
27 #= '/*========*/'
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
28 # Blank line
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
29
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
30 - Parameter row:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
31 @Parameter,Value [, ...]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
32 Supported Parameter terms:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
33 standard Standard name. E.g. '3GPP TS 29.272', 'RFC 6733'.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
34 vendor Vendor number.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
35
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
36 """
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
37
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
38 from __future__ import print_function
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
39 from __future__ import with_statement
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
40
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
41 import abc
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
42 import csv
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
43 import collections
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
44 import json
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
45 import re
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
46 import optparse
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
47 import os
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
48 import sys
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
49
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
50 CSV_COLUMN_NAMES = [
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
51 'name',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
52 'code',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
53 'section',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
54 'datatype',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
55 'must',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
56 'may',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
57 'shouldnot',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
58 'mustnot',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
59 ]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
60
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
61 DERIVED_TO_BASE = {
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
62 'Address': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
63 'Time': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
64 'UTF8String': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
65 'DiameterIdentity': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
66 'DiameterURI': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
67 'Enumerated': 'Integer32', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
68 'IPFilterRule': 'OctetString', # RFC 6733 section 4.3.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
69 'QoSFilterRule': 'OctetString', # RFC 7155 section 4.1.1
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
70 }
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
71
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
72 # See https://www.iana.org/assignments/enterprise-numbers/enterprise-numbers
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
73 VENDOR_TO_NAME = {
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
74 0: '',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
75 193: 'Ericsson',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
76 8164: 'Starent',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
77 10415: '3GPP',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
78 }
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
79
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
80
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
81 class Avp(object):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
82 """Store an AVP row."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
83
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
84 # Regex to validate avp-name per RFC 6733 section 3.2,
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
85 # with changes:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
86 # - Allow avp-name to start with numbers (for 3GPP)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
87 # - Allow '.' in avp-name, for existing dict_dcca_3gpp usage.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
88 # TODO: if starts with digit, ensure contains a letter somewhere?
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
89 _name_re = re.compile(r'^[a-zA-Z0-9][a-zA-Z0-9-\.]*$')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
90
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
91 # Regex to validate flags: M, P, V, comma, space
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
92 _flags_re = re.compile(r'^[MPV, ]*$')
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
93
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
94 __slots__ = CSV_COLUMN_NAMES + [
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
95 'filename', 'line_num', 'standard', 'vendor', ]
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
96
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
97 def __init__(self, name, code, section, datatype,
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
98 must, may, shouldnot, mustnot, extra_cells=[],
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
99 filename='', line_num=0, standard='', vendor=0):
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
100 # Members from CSV row
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
101 self.name = name
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
102 self.code = int(code)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
103 self.section = section
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
104 self.datatype = datatype
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
105 self.must = must
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
106 self.may = may
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
107 self.shouldnot = shouldnot
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
108 self.mustnot = mustnot
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
109 # Members from file state
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
110 self.filename = filename
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
111 self.line_num = line_num
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
112 self.standard = standard
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
113 self.vendor = vendor
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
114 # Validate CSV fields
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
115 if not self._name_re.match(self.name):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
116 raise ValueError('Invalid AVP name "{}"'.format(self.name))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
117 if (self.code < 0 or self.code > 4294967295):
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
118 raise ValueError('AVP "{}" invalid code {}'.format(
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
119 self.name, self.code))
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
120 if (self.datatype not in (
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
121 'OctetString', 'Integer32', 'Integer64', 'Unsigned32',
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
122 'Unsigned64', 'Float32', 'Float64', 'Grouped')
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
123 and self.datatype not in DERIVED_TO_BASE):
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
124 raise ValueError('{} invalid data type "{}"'.format(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
125 self.description(), self.datatype))
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
126 # Validate flags
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
127 flags = collections.Counter()
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
128 for val, desc in [
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
129 (self.must, 'Must'),
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
130 (self.may, 'May'),
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
131 (self.shouldnot, 'Should Not'),
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
132 (self.mustnot, 'Must Not'),
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
133 ]:
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
134 if not self._flags_re.match(val):
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
135 raise ValueError('{} invalid {} Flags "{}"'.format(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
136 self.description(), desc, val))
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
137 flags.update(val)
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
138 # Check occurrence of M,V in Must,May,ShouldNot,MustNot
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
139 for flag in 'MV':
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
140 # TODO: can AVP flags not appear at all?
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
141 # if flags[flag] == 0:
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
142 # raise ValueError('{} Flag "{}" not set'.format(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
143 # self.description(), flag))
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
144 if flags[flag] > 1:
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
145 raise ValueError('{} Flag "{}" set {} times'.format(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
146 self.description(), flag, flags[flag]))
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
147 # Compare V presence against vendor
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
148 if 'V' in self.must:
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
149 if self.vendor == 0:
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
150 raise ValueError('{} Flag "V" set for vendor 0'.format(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
151 self.description()))
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
152 else:
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
153 if self.vendor != 0:
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
154 raise ValueError('{} Flag "V" not set for vendor {}'.format(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
155 self.description(), self.vendor))
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
156
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
157 @property
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
158 def __dict__(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
159 return {s: getattr(self, s) for s in self.__slots__}
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
160
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
161 def __eq__(self, other):
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
162 """Equality comparison of Avp instances.
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
163 Considered equal if name, vendor, code, datatype, and flags are equal.
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
164 """
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
165 if other is self:
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
166 return True
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
167 if type(other) is not type(self):
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
168 return NotImplemented
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
169 return (
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
170 other.name, other.vendor, other.code, other.datatype,
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
171 other.must, other.may, other.shouldnot, other.mustnot,
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
172 ) == (
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
173 self.name, self.vendor, self.code, self.datatype,
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
174 self.must, self.may, self.shouldnot, self.mustnot,
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
175 )
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
176
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
177 def __ne__(self, other):
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
178 return not self == other
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
179
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
180 def description(self):
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
181 return 'AVP "{}" ({})'.format(self.name, self.code)
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
182
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
183
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
184 class Processor(object):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
185 """Interface for processor of Avp"""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
186
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
187 __metaclass__ = abc.ABCMeta
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
188
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
189 @classmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
190 def cls_name(cls):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
191 """Return the name, lower-case, without "processor" suffix."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
192 suffix = 'processor'
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
193 name = cls.__name__.lower()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
194 if name.endswith(suffix):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
195 return name[:-len(suffix)]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
196 return name
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
197
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
198 @classmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
199 def cls_desc(cls):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
200 """Return the first line of the docstring."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
201 if cls.__doc__ is None:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
202 return ""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
203 return cls.__doc__.split('\n')[0]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
204
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
205 @abc.abstractmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
206 def next_file(self, filename):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
207 """Called when a file is opened."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
208 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
209
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
210 @abc.abstractmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
211 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
212 """Process a validated Avp."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
213 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
214
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
215 @abc.abstractmethod
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
216 def comment(self, comment, filename, line_num):
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
217 """Process a comment row:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
218 #comment,
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
219 """
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
220 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
221
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
222 @abc.abstractmethod
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
223 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
224 """Invoked after all rows processed."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
225 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
226
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
227
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
228 class DebugProcessor(Processor):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
229 """Display the CSV parsing"""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
230
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
231 def next_file(self, filename):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
232 print('File: {}'.format(filename))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
233
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
234 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
235 avpdict = vars(avp)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
236 print('AVP: {name}, {code}, {datatype}'.format(**avpdict))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
237
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
238 def comment(self, comment, filename, line_num):
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
239 print('Comment: {}'.format(comment))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
240
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
241 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
242 print('Generate')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
243
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
244
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
245 class NoopProcessor(Processor):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
246 """Validate the CSV; no other output"""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
247
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
248 def next_file(self, filename):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
249 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
250
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
251 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
252 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
253
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
254 def comment(self, comment, filename, line_num):
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
255 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
256
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
257 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
258 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
259
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
260
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
261 class FdcProcessor(Processor):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
262 """Generate freeDiameter C code
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
263
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
264 Comment cells are parsed as:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
265 # text comment /* text comment */
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
266 #= /*==============*/
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
267 # [blank line]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
268 """
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
269
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
270 COMMENT_WIDTH = 64
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
271
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
272 def __init__(self):
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
273 self.filenames = []
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
274 self.lines = []
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
275
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
276 def next_file(self, filename):
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
277 self.filenames.append(os.path.basename(filename))
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
278
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
279 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
280 comment = '{name}, {datatype}, code {code}'.format(**vars(avp))
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
281 if avp.section != '':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
282 comment += ', section {}'.format(avp.section)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
283 self.add_comment(comment)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
284 self.add('\t{')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
285 self.add('\t\tstruct dict_avp_data data = {')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
286 # TODO: remove comments?
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
287 self.add('\t\t\t{},\t/* Code */'.format(avp.code))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
288 self.add('\t\t\t{},\t/* Vendor */'.format(avp.vendor))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
289 self.add('\t\t\t\"{}\",\t/* Name */'.format(avp.name))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
290 self.add('\t\t\t{},\t/* Fixed flags */'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
291 self.build_flags(', '.join([avp.must, avp.mustnot]))))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
292 self.add('\t\t\t{},\t/* Fixed flag values */'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
293 self.build_flags(avp.must)))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
294 # TODO: add trailing comma?
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
295 self.add('\t\t\tAVP_TYPE_{}\t/* base type of data */'.format(
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
296 DERIVED_TO_BASE.get(avp.datatype, avp.datatype).upper()))
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
297 self.add('\t\t};')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
298 avp_type = 'NULL'
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
299 if avp.datatype == 'Enumerated':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
300 self.add('\t\tstruct dict_object\t*type;')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
301 vendor_prefix = ''
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
302 if avp.vendor != 0:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
303 vendor_prefix = '{}/'.format(VENDOR_TO_NAME[avp.vendor])
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
304 self.add(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
305 '\t\tstruct dict_type_data\t tdata = {{ AVP_TYPE_INTEGER32, '
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
306 '"Enumerated({prefix}{name})", NULL, NULL, NULL }};'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
307 prefix=vendor_prefix, name=avp.name))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
308 # XXX: add enumerated values
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
309 self.add('\t\tCHECK_dict_new(DICT_TYPE, &tdata, NULL, &type);')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
310 avp_type = "type"
1463
8f6c77f24b1a csv_to_fd: add QoSFilterRule. style fixes
Luke Mewburn <luke@mewburn.net>
parents: 1461
diff changeset
311 elif avp.datatype in DERIVED_TO_BASE:
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
312 avp_type = '{}_type'.format(avp.datatype)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
313 self.add('\t\tCHECK_dict_new(DICT_AVP, &data, {}, NULL);'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
314 avp_type))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
315 # TODO: remove ; on scope brace
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
316 self.add('\t};')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
317 self.add('')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
318
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
319 def comment(self, comment, filename, line_num):
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
320 if comment == '':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
321 self.add('')
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
322 elif comment == '=':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
323 self.add_header()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
324 elif comment.startswith(' '):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
325 self.add_comment(comment[1:])
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
326 else:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
327 raise ValueError('Unsupported comment "{}"'.format(comment))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
328
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
329 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
330 self.print_header()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
331 self.print_comment('Start of generated data.')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
332 self.print_comment('')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
333 self.print_comment('The following is created automatically with:')
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
334 self.print_comment(' csv_to_fd -p {} {}'.format(
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
335 self.cls_name(), ' '.join(self.filenames)))
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
336 self.print_comment('Changes will be lost during the next update.')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
337 self.print_comment('Do not modify;'
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
338 ' modify the source .csv file instead.')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
339 self.print_header()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
340 print('')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
341 print('\n'.join(self.lines))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
342 self.print_header()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
343 self.print_comment('End of generated data.')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
344 self.print_header()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
345
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
346 def build_flags(self, flags):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
347 result = []
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
348 if 'V' in flags:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
349 result.append('AVP_FLAG_VENDOR')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
350 if 'M' in flags:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
351 result.append('AVP_FLAG_MANDATORY')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
352 return ' |'.join(result)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
353
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
354 def add(self, line):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
355 self.lines.append(line)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
356
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
357 def add_comment(self, comment):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
358 self.lines.append(self.format_comment(comment))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
359
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
360 def add_header(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
361 self.lines.append(self.format_header())
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
362
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
363 def format_comment(self, comment):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
364 return '\t/* {:<{width}} */'.format(comment, width=self.COMMENT_WIDTH)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
365
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
366 def format_header(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
367 return '\t/*={:=<{width}}=*/'.format('', width=self.COMMENT_WIDTH)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
368
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
369 def print_comment(self, comment):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
370 print(self.format_comment(comment))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
371
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
372 def print_header(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
373 print(self.format_header())
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
374
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
375
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
376 class JsonProcessor(Processor):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
377 """Generate freeDiameter JSON object
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
378 """
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
379
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
380 def __init__(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
381 self.avps = []
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
382
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
383 def next_file(self, filename):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
384 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
385
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
386 def avp(self, avp):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
387 flags = collections.OrderedDict([
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
388 ('Must', self.build_flags(avp.must)),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
389 ('MustNot', self.build_flags(avp.mustnot)),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
390 ])
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
391 row = collections.OrderedDict([
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
392 ('Code', avp.code),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
393 ('Flags', flags),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
394 ('Name', avp.name),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
395 ('Type', avp.datatype),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
396 ('Vendor', avp.vendor),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
397 ])
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
398 self.avps.append(row)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
399
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
400 def comment(self, comment, filename, line_num):
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
401 pass
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
402
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
403 def generate(self):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
404 doc = {"AVPs": self.avps}
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
405 print(json.dumps(doc, indent=2))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
406
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
407 def build_flags(self, flags):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
408 result = []
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
409 if 'V' in flags:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
410 result.append('V')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
411 if 'M' in flags:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
412 result.append('M')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
413 return ''.join(result)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
414
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
415
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
416 def avp_conflict(description, avp, conflict):
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
417 """Raise error for duplicate or conflicting AVPs.
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
418 """
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
419 if avp == conflict:
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
420 raise ValueError(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
421 '{} {} duplicated in'
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
422 ' file "{}" line {}'.format(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
423 avp.description(), description,
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
424 conflict.filename, conflict.line_num))
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
425 else:
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
426 raise ValueError(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
427 '{} {} conflicts with {}'
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
428 ' in file "{}" line {}'.format(
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
429 avp.description(), description,
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
430 conflict.description(),
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
431 conflict.filename, conflict.line_num))
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
432
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
433
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
434 def main():
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
435 """Main application entry.
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
436 """
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
437
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
438 # Build dict of name: NameProcessor
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
439 processors = {
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
440 cls.cls_name(): cls
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
441 for cls in Processor.__subclasses__()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
442 }
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
443
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
444 # Build Processor name to desc
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
445 processor_help = '\n'.join(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
446 [' {:8} {}'.format(key, processors[key].cls_desc())
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
447 for key in sorted(processors)])
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
448
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
449 # Custom OptionParser with improved help
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
450 class MyParser(optparse.OptionParser):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
451 """Custom OptionParser without epilog formatting."""
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
452 def format_help(self, formatter=None):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
453 return """\
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
454 {}
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
455 Supported PROCESSOR options:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
456 {}
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
457 """.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
458 optparse.OptionParser.format_help(self, formatter),
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
459 processor_help)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
460
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
461 # Parse options
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
462 parser = MyParser(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
463 description="""\
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
464 Convert CSV files containing RADIUS or Diameter AVP tables
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
465 into various formats using the specified processor PROCESSOR.
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
466 """)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
467
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
468 parser.add_option(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
469 '-p', '--processor',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
470 default='noop',
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
471 help='AVP processor. One of: {}. [%default]'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
472 ', '.join(processors.keys())))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
473 (opts, args) = parser.parse_args()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
474 if len(args) < 1:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
475 parser.error('Incorrect number of arguments')
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
476
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
477 # Find processor
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
478 try:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
479 avpproc = processors[opts.processor]()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
480 except KeyError as e:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
481 parser.error('Unknown processor "{}"'.format(opts.processor))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
482
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
483 # dict of [vendor][code] : Avp
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
484 avp_codes = collections.defaultdict(dict)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
485
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
486 # dict of [vendor][name] : Avp
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
487 avp_names = collections.defaultdict(dict)
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
488
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
489 # Process files
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
490 for filename in args:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
491 avpproc.next_file(filename)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
492 with open(filename, 'r') as csvfile:
1469
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
493 csvdata = csv.DictReader(csvfile, CSV_COLUMN_NAMES,
48fa8d70e6ad csv_to_fd: validate flags and other improvements
Luke Mewburn <luke@mewburn.net>
parents: 1464
diff changeset
494 restkey='extra_cells')
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
495 standard = ''
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
496 vendor = 0
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
497 errors = []
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
498 for row in csvdata:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
499 try:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
500 if row['name'] in (None, '', 'Attribute Name'):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
501 continue
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
502 elif row['name'].startswith('#'):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
503 comment = row['name'][1:]
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
504 avpproc.comment(comment, filename, csvdata.line_num)
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
505 elif row['name'].startswith('@'):
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
506 parameter = row['name'][1:]
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
507 value = row['code']
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
508 if False:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
509 pass
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
510 elif parameter == 'standard':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
511 standard = value
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
512 elif parameter == 'vendor':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
513 vendor = int(value)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
514 else:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
515 raise ValueError('Unknown parameter "{}"'.format(
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
516 parameter))
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
517 else:
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
518 avp = Avp(filename=filename, line_num=csvdata.line_num,
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
519 standard=standard, vendor=vendor,
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
520 **row)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
521 # Ensure AVP vendor/code not already defined
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
522 if avp.code in avp_codes[avp.vendor]:
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
523 conflict = avp_codes[avp.vendor][avp.code]
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
524 avp_conflict('Code', avp, conflict)
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
525 avp_codes[avp.vendor][avp.code] = avp
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
526 # Ensure AVP vendor/name not already defined
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
527 if avp.name in avp_names[avp.vendor]:
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
528 conflict = avp_names[avp.vendor][avp.name]
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
529 avp_conflict('Name', avp, conflict)
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
530 avp_names[avp.vendor][avp.name] = avp
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
531 # Process AVP
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
532 avpproc.avp(avp)
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
533 except ValueError as e:
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
534 errors.append('CSV file "{}" line {}: {}\n'.format(
1479
c0aa1e66c12e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1469
diff changeset
535 filename, csvdata.line_num, e))
1487
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
536 if errors:
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
537 sys.stderr.write(''.join(errors))
5c2d061a8c8e csv_to_fd: improve validation
Luke Mewburn <luke@mewburn.net>
parents: 1479
diff changeset
538 sys.exit(1)
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
539
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
540 # Generate result
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
541 avpproc.generate()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
542
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
543
1464
1404de313b85 csv_to_fd: consistent comparison order
Luke Mewburn <luke@mewburn.net>
parents: 1463
diff changeset
544 if __name__ == '__main__':
1461
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
545 main()
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
546
a86eb3375b95 add csv_to_fd, org_to_csv tools
Luke Mewburn <luke@mewburn.net>
parents:
diff changeset
547 # vim: set et sw=4 sts=4 :
"Welcome to our mercurial repository"