R
Reed Kimble
Guest
Until recently, I hadnt had a need to utilize JSON so I dont have a lot of experience with it. But JSON has become the standard for web transactions (displacing XML), and its used in a lot of places for a lot of purposes so Im having to come to terms with using it.
I believe the common standard for handling JSON in .Net is to use Newtonsoft.JSON, and it is great at what it does. When you know the object model which the JSON represents and have (or can easily construct) the POCO for that model, then it is really fast and easy to deserialize the JSON into a graph of object instances from the model, or query the model via LINQ.
But when you dont know what the model looks like, or dont have (or dont want to create) the plain-old-class-objects representing the model, then Newtonsoft.JSON can be a bit of a pain. Examining the hierarchy of the object model parsed by Newtonsoft.JSON is not intuitive and does not explicitly follow the JSON document format. These arent problems or criticisms; Netwtonsoft.JSON was simply not built for this purpose.
To address this, Ive built a small library Im calling "JsonCore" which follows the JSON document structure and provides simplified parsing of JSON document text into JsonValue objects.
A JsonValue object is the base object type of a JSON data element. It can represent one of a String, Number, Object, Array, Boolean, or Null as per the JSON data specification. A JsonValue instance contains IsDataType() methods which can be called to interrogate the instance and determine the JSON data element it represents. There are also associated AsDataType() methods to cast the JsonValue instance into the specific type, if the JsonValue instance represents an instance of the related JSON data element (if the data type does not match the return type of the method called, an InvalidOperationException will occur).
A JsonValue with the data type Array can be cast into a JsonArray instance. The JsonArray object will provide access to the collection of JsonValue objects contained within the JsonArray.
A JsonValue with the data type Object can be cast into a JsonObject instance. The JsonObject object will provide access to the named properties contained within the JsonObject. The object can also be enumerated as a collection of key-value-pairs of string and JsonValue.
This design makes it very easy to parse a JSON string and then analyze the resulting document hierarchy. The debug display in Visual Studio is able to provide a clean and structured view into the parsed object model. The resulting collection of JsonValue objects is also easy to navigate and consume with code.
Ive only implemented the consumption of JSON so far, not its generation, but this was the hard part; generating JSON is much easier and Ill probably add that functionality to the library in the future. Right now I want to concentrate on the parsing to ensure that the specification has been accurately followed and that there are no bugs in the implementation. This iteration of the code has executed quickly and accurately against three separate sources that Ive tested. I believe it is correct and bug-free, but you never know... Id love to get some feedback on how this performs for other JSON sources, and usability suggestions are welcome as well.
The library is presented in two code files; one to contain the JsonValue object definitions and one to contain the JsonParser implementation. First the object definitions. This code looks long, but it is mostly boilerplate. The objects themselves are actually quite simple.
Option Strict On
Namespace JsonCore
<summary>
Represents a JsonValue of type Array; one of the two fundemental JSON structures (Object and Array).
</summary>
Public Class JsonArray
Inherits JsonValue
Implements IEnumerable(Of JsonValue)
Public Shared Function FromJsonValue(value As JsonValue) As JsonArray
If value.IsArray Then Return CType(value, JsonArray)
Dim result = New JsonArray()
result.items.Add(value)
Return result
End Function
Private items As List(Of JsonValue)
<summary>
Creates a new, empty JsonArray instance.
</summary>
Public Sub New()
MyBase.New(JsonDataType.Array, Nothing)
items = DirectCast(_Value, List(Of JsonValue))
End Sub
<summary>
Gets the number of JsonValue objects contianed within this JsonArray.
</summary>
<returns>An Integer representing the number of JsonValues in the array.</returns>
Public ReadOnly Property Count As Integer
Get
Return items.Count
End Get
End Property
<summary>
Gets or sets the JsonValue at the specified index in this JsonArray.
</summary>
<param name="index">The index of the JsonValue to get or set.</param>
<returns>The JsonValue at the specified index.</returns>
Default Public Property Item(index As Integer) As JsonValue
Get
Return items(index)
End Get
Set(value As JsonValue)
items(index) = value
End Set
End Property
<summary>
Adds a new JsonValue instance to the end of this JsonArray.
</summary>
<param name="value">The JsonValue to append to the end of the array.</param>
Public Sub Push(value As JsonValue)
items.Add(value)
End Sub
<summary>
Gets an enumerator of JsonValues contained within this JsonArray.
</summary>
<returns>An enumerator of JsonValues contained within this JsonArray.</returns>
Public Function GetEnumerator() As IEnumerator(Of JsonValue) Implements IEnumerable(Of JsonValue).GetEnumerator
Return items.GetEnumerator
End Function
Private Function IEnumerable_GetEnumerator() As IEnumerator Implements IEnumerable.GetEnumerator
Return GetEnumerator()
End Function
End Class
<summary>
Represents a JsonValue of type Object; one of the two fundemental JSON structures (Object and Array).
</summary>
Public Class JsonObject
Inherits JsonValue
Implements IEnumerable(Of KeyValuePair(Of String, JsonValue))
Private properties As Dictionary(Of String, JsonValue)
<summary>
Gets or sets a named property value on this JsonObject.
</summary>
<param name="name">The name of the property to get or set.</param>
<returns>A JsonValue representing the value of the specified property.</returns>
Default Public Property [Property](name As String) As JsonValue
Get
Return properties(name)
End Get
Set(value As JsonValue)
properties(name) = value
End Set
End Property
<summary>
Gets the number of properties on this JsonObject.
</summary>
<returns>An Integer representing the number of properties on this JsonObject.</returns>
Public ReadOnly Property PropertyCount As Integer
Get
Return properties.Count
End Get
End Property
<summary>
Gets the property name at the specified index, in the order the properties were discovered.
</summary>
<param name="index">The index of the property name to get.</param>
<returns>A String representing the property name at the specified index.</returns>
Public ReadOnly Property PropertyName(index As Integer) As String
Get
Return properties.Keys.ElementAt(index)
End Get
End Property
<summary>
Gets the collection of property names for this JsonObject.
</summary>
<returns>An IEnumerable of Strings containing the property names defined in this JsonObject.</returns>
Public ReadOnly Property PropertyNames As IEnumerable(Of String)
Get
Return properties.Keys
End Get
End Property
<summary>
Creates a new JsonObject instance with no defined properties.
</summary>
Public Sub New()
MyBase.New(JsonDataType.Object, Nothing)
properties = DirectCast(_Value, Dictionary(Of String, JsonValue))
End Sub
<summary>
Gets an enumerator of key/value pairs of Strings and JsonValues, representing the properties of this JsonObject.
</summary>
<returns>An enumerator of key/value pairs of Strings and JsonValues representing the properties of this JsonObject.</returns>
Public Function GetEnumerator() As IEnumerator(Of KeyValuePair(Of String, JsonValue)) Implements IEnumerable(Of KeyValuePair(Of String, JsonValue)).GetEnumerator
Return properties.GetEnumerator
End Function
Private Function IEnumerable_GetEnumerator() As IEnumerator Implements IEnumerable.GetEnumerator
Return GetEnumerator()
End Function
End Class
<summary>
Represents a value in a JSON document of a particular data type. Valid JSON data types are String, Number, Object,
Array, Boolean, and Null. Strings may have an extended type of Date, and Numbers will have an extended type of
either Integer or Decimal.
</summary>
Public Class JsonValue
Protected _Value As Object
<summary>
Gets a value from the JsonDataType enum representing the JSON data type for this value.
</summary>
<returns></returns>
Public ReadOnly Property ValueType As JsonDataType
<summary>
Gets the extended data type (Date, Integer, or Decimal) for the value, if applicable.
</summary>
<returns></returns>
Public ReadOnly Property ExtendedType As JsonExtendedType
<summary>
Creates a new Null JsonValue instance.
</summary>
Public Sub New()
Me.New(JsonDataType.Null, Nothing)
End Sub
<summary>
Creates a new Number JsonValue instance with the extended type Decimal.
</summary>
<param name="value">The Decimal value assigned to this JsonValue instance.</param>
Public Sub New(value As Decimal)
Me.New(JsonExtendedType.Decimal, value)
End Sub
<summary>
Creates a new Number JsonValue instance with the extended type Integer.
</summary>
<param name="value">The Integer value assigned to this JsonValue instance.</param>
Public Sub New(value As Integer)
Me.New(JsonExtendedType.Integer, value)
End Sub
<summary>
Creates a new String JsonValue instance.
</summary>
<param name="value">The String value assigned to this JsonValue instance.</param>
Public Sub New(value As String)
Me.New(JsonDataType.String, value)
End Sub
<summary>
Creates a new String JsonValue instance with the extended type Date.
</summary>
<param name="value">The Date value assigned to this JsonValue instance.</param>
Public Sub New(value As Date)
Me.New(JsonExtendedType.Date, value)
End Sub
<summary>
Creates a new Boolean JsonValue instance.
</summary>
<param name="value">The Boolean value assigned to this JsonValue instance.</param>
Public Sub New(value As Boolean)
Me.New(JsonDataType.Boolean, value)
End Sub
Protected Sub New(type As JsonDataType, value As Object)
ValueType = type
Initialize(value)
End Sub
Protected Sub New(type As JsonExtendedType, value As Object)
Select Case type
Case JsonExtendedType.Date, JsonExtendedType.None
ValueType = JsonDataType.String
Case JsonExtendedType.Decimal, JsonExtendedType.Integer
ValueType = JsonDataType.Number
End Select
ExtendedType = type
Initialize(value)
End Sub
Protected Sub Initialize(value As Object)
Select Case ValueType
Case JsonDataType.Array
_Value = New List(Of JsonValue)
If TypeOf value Is IEnumerable(Of JsonValue) Then
DirectCast(_Value, List(Of JsonValue)).AddRange(DirectCast(value, IEnumerable(Of JsonValue)))
ElseIf TypeOf value Is JsonValue
DirectCast(_Value, List(Of JsonValue)).Add(DirectCast(value, JsonValue))
End If
Case JsonDataType.Object
If TypeOf value Is IDictionary(Of String, JsonValue) Then
_Value = New Dictionary(Of String, JsonValue)(DirectCast(value, IDictionary(Of String, JsonValue)))
Else
_Value = New Dictionary(Of String, JsonValue)
If value?.GetType.IsArray Then
For Each entry In CType(value, Array)
If entry.GetType.IsArray Then
Dim values = CType(entry, Array)
If values.Length = 2 Then
If TypeOf values.GetValue(0) Is String Then
If TypeOf values.GetValue(1) Is JsonValue Then
DirectCast(_Value, Dictionary(Of String, JsonValue)).Add(DirectCast(values.GetValue(0), String), DirectCast(values.GetValue(1), JsonValue))
End If
End If
End If
End If
Next
End If
End If
Case JsonDataType.Boolean
_Value = CBool(value)
Case JsonDataType.Null
_Value = Nothing
Case JsonDataType.Number
Select Case ExtendedType
Case JsonExtendedType.Integer
_Value = CInt(value)
Case JsonExtendedType.Decimal
_Value = CDec(value)
End Select
Case JsonDataType.String
_Value = CStr(value)
Case Else
_Value = value
End Select
End Sub
<summary>
Gets this JsonValue instance as a JsonArray object, if the instance represents a JsonArray.
Otherwise throws an InvalidOperationException.
</summary>
<returns>A JsonArray instance representing this JsonValue.</returns>
Public Function AsArray() As JsonArray
If ValueType = JsonDataType.Array Then Return DirectCast(Me, JsonArray)
Throw New InvalidOperationException("JsonDataType is not Array.")
End Function
<summary>
Gets this JsonValue instance as a Boolean object, if the instance represents a Boolean.
Otherwise throws an InvalidOperationException.
</summary>
<returns>A Boolean instance representing this JsonValue.</returns>
Public Function AsBoolean() As Boolean
If ValueType = JsonDataType.Boolean Then Return DirectCast(_Value, Boolean)
Throw New InvalidOperationException("JsonDataType is not Boolean.")
End Function
<summary>
Gets this JsonValue instance as a Date object, if the instance represents a Date.
Otherwise throws an InvalidOperationException.
</summary>
<returns>A Date instance representing this JsonValue.</returns>
Public Function AsDate() As Date
If ExtendedType = JsonExtendedType.Date Then Return DirectCast(_Value, Date)
Throw New InvalidOperationException("JsonExtendedType is not Date.")
End Function
<summary>
Gets this JsonValue instance as a Decimal object, if the instance represents a Decimal.
Otherwise throws an InvalidOperationException.
</summary>
<returns>A Decimal instance representing this JsonValue.</returns>
Public Function AsDecimal() As Decimal
If ExtendedType = JsonExtendedType.Decimal Then Return DirectCast(_Value, Decimal)
Throw New InvalidOperationException("JsonExtendedType is not Decimal.")
End Function
<summary>
Gets this JsonValue instance as an Integer object, if the instance represents an Integer.
Otherwise throws an InvalidOperationException.
</summary>
<returns>An Integer instance representing this JsonValue.</returns>
Public Function AsInteger() As Integer
If ExtendedType = JsonExtendedType.Integer Then Return DirectCast(_Value, Integer)
Throw New InvalidOperationException("JsonExtendedType is not Integer.")
End Function
<summary>
Gets this JsonValue instance as a JsonObject object, if the instance represents a JsonObject.
Otherwise throws an InvalidOperationException.
</summary>
<returns>A JsonObject instance representing this JsonValue.</returns>
Public Function AsObject() As JsonObject
If ValueType = JsonDataType.Object Then Return DirectCast(Me, JsonObject)
Throw New InvalidOperationException("JsonDataType is not Object.")
End Function
<summary>
Gets this JsonValue instance as a String object, if the instance represents a String.
Otherwise throws an InvalidOperationException.
</summary>
<returns>A String instance representing this JsonValue.</returns>
Public Function AsString() As String
If ValueType = JsonDataType.String Then Return DirectCast(_Value, String)
Throw New InvalidOperationException("JsonDataType is not String.")
End Function
Public Overrides Function Equals(obj As Object) As Boolean
If TypeOf obj Is JsonValue Then
Dim other = DirectCast(obj, JsonValue)
If ValueType = other.ValueType Then
Return Comparer.Default.Compare(_Value, other._Value) = 0
End If
End If
Return False
End Function
<summary>
Gets a value indicating whether or not this JsonValue instance represents a JsonArray.
</summary>
<returns>True if this JsonValue is a JsonArray, otherwise false.</returns>
Public Function IsArray() As Boolean
Return ValueType = JsonDataType.Array
End Function
<summary>
Gets a value indicating whether or not this JsonValue instance represents a Boolean.
</summary>
<returns>True if this JsonValue is a Boolean, otherwise false.</returns>
Public Function IsBoolean() As Boolean
Return ValueType = JsonDataType.Boolean
End Function
<summary>
Gets a value indicating whether or not this JsonValue instance represents a Date.
</summary>
<returns>True if this JsonValue is a Date, otherwise false.</returns>
Public Function IsDate() As Boolean
Return ExtendedType = JsonExtendedType.Date
End Function
<summary>
Gets a value indicating whether or not this JsonValue instance represents a Decimal.
</summary>
<returns>True if this JsonValue is a Decimal, otherwise false.</returns>
Public Function IsDecimal() As Boolean
Return ExtendedType = JsonExtendedType.Decimal
End Function
<summary>
Gets a value indicating whether or not this JsonValue instance represents an Integer.
</summary>
<returns>True if this JsonValue is an Integer, otherwise false.</returns>
Public Function IsInteger() As Boolean
Return ExtendedType = JsonExtendedType.Integer
End Function
<summary>
Gets a value indicating whether or not this JsonValue instance represents Null.
</summary>
<returns>True if this JsonValue is Null, otherwise false.</returns>
Public Function IsNull() As Boolean
Return ValueType = JsonDataType.Null
End Function
<summary>
Gets a value indicating whether or not this JsonValue instance represents a JsonObject.
</summary>
<returns>True if this JsonValue is a JsonObject, otherwise false.</returns>
Public Function IsObject() As Boolean
Return ValueType = JsonDataType.Object
End Function
<summary>
Gets a value indicating whether or not this JsonValue instance represents a String.
</summary>
<returns>True if this JsonValue is a String, otherwise false.</returns>
Public Function IsString() As Boolean
Return ValueType = JsonDataType.String
End Function
Public Overloads Function ReferenceEquals(other As Object) As Boolean
Return Me Is other
End Function
Public Overrides Function ToString() As String
If _Value Is Nothing Then Return String.Empty
Return _Value.ToString
End Function
End Class
<summary>
Specifies the JSON data type of a JsonValue
</summary>
Public Enum JsonDataType
Null
[String]
Number
[Boolean]
Array
[Object]
End Enum
<summary>
Specifies the extended data type of a JSON String or Number.
</summary>
Public Enum JsonExtendedType
None
[Date]
[Decimal]
[Integer]
End Enum
End Namespace
Next comes the parser implementation and related code files. This is the complex part. The API is fully commented and there are a few code comments sprinkled throughout to help describe the purpose of the various code blocks, so hopefully it isnt too terribly difficult to follow along and understand how the parsing engine works. The JSON specification is pretty simple so it doesnt require a very complex parser. I tried to create a robust parsing engine... hopefully it is not overcomplicated.
Option Strict On
Namespace JsonCore
<summary>
Parses a JSON document string into a collection of JsonValue objects.
</summary>
Public Class JsonParser
<summary>
Creates a collection of JsonValue objects from the given JSON text string.
</summary>
<param name="text">The JSON string to parse into a JsonValue collection.</param>
<returns>An IEnumerable instance of JsonValue objects parsed from the supplied string.</returns>
Public Shared Function Parse(text As String) As IEnumerable(Of JsonValue)
Dim result As New List(Of JsonValue) holds the final, parsed JsonValue instances to return
Dim entities As New Stack(Of JEntity) stores the in-process entities being parsed
Dim tokens As List(Of Token) = Lex(text) hold the list of tokens returned from the lexer
Dim processClosure = Sub(value As Object) define helper to generate JsonValue instances from parsed entities
Dim topEntity = entities.Peek
Select Case topEntity.Kind
Case TokenKind.OpenArray
If TypeOf value Is JsonValue Then
DirectCast(topEntity.Value, JsonArray).Push(DirectCast(value, JsonValue))
Else
DirectCast(topEntity.Value, JsonArray).Push(MakeJsonStringValue(DirectCast(value, String)))
End If
Case TokenKind.DelimitEntry
entities.Pop()
topEntity = entities.Peek
If topEntity.Kind = TokenKind.OpenArray Then
If TypeOf value Is JsonValue Then
DirectCast(topEntity.Value, JsonArray).Push(DirectCast(value, JsonValue))
Else
DirectCast(topEntity.Value, JsonArray).Push(MakeJsonStringValue(DirectCast(value, String)))
End If
ElseIf topEntity.Kind = TokenKind.OpenObject
entities.Push(New JEntity(TokenKind.Text, value))
End If
Case TokenKind.OpenObject
entities.Push(New JEntity(TokenKind.Text, value))
Case TokenKind.DelimitProperty
entities.Pop()
Dim key = entities.Pop.Value.ToString
topEntity = entities.Peek
If TypeOf value Is JsonValue Then
DirectCast(topEntity.Value, JsonObject).Property(key) = DirectCast(value, JsonValue)
Else
DirectCast(topEntity.Value, JsonObject).Property(key) = MakeJsonStringValue(DirectCast(value, String))
End If
End Select
End Sub
Dim index As Integer index used to loop over token collection
Try
For index = 0 To tokens.Count - 1 Parse the token list into entities
Dim thisToken = tokens(index)
Select Case thisToken.Kind
Case TokenKind.OpenArray
entities.Push(New JEntity(TokenKind.OpenArray, New JsonArray))
Case TokenKind.OpenObject
entities.Push(New JEntity(TokenKind.OpenObject, New JsonObject))
Case TokenKind.CloseArray
Dim closedArray = DirectCast(entities.Pop.Value, JsonArray)
If entities.Count > 0 Then
processClosure(closedArray)
Else
result.Add(closedArray)
End If
Case TokenKind.CloseObject
Dim closedObject = DirectCast(entities.Pop.Value, JsonObject)
If entities.Count > 0 Then
processClosure(closedObject)
Else
result.Add(closedObject)
End If
Case TokenKind.Text
processClosure(UnescapeString(thisToken, text))
Case TokenKind.Number
processClosure(MakeJsonNumberValue(text.Substring(thisToken.Index, thisToken.Length)))
Case TokenKind.Bool
processClosure(MakeJsonBoolValue(text.Substring(thisToken.Index, thisToken.Length)))
Case TokenKind.Null
processClosure(New JsonValue())
Case TokenKind.DelimitProperty
entities.Push(New JEntity(TokenKind.DelimitProperty, Nothing))
Case TokenKind.DelimitEntry
entities.Push(New JEntity(TokenKind.DelimitEntry, Nothing))
End Select
Next
Catch ex As Exception
Throw New ParsingException(index, tokens(index).Kind, tokens.ElementAtOrDefault(index - 1).Kind, ex)
End Try
Return result.AsEnumerable
End Function
<summary>
Aids the lexer in finding each tokens start index and length within the source JSON string.
</summary>
<param name="text">The raw JSON text string.</param>
<param name="index">The current index being analyzed within the raw string.</param>
<param name="length">(OUT) Returns the length of the discovered token within the raw string.</param>
<returns>A value from the TokenKind enum representing the type of token discovered.</returns>
Private Shared Function FindToken(text As String, ByRef index As Integer, ByRef length As Integer) As TokenKind
helper for matching TRUE/FALSE/NULL values
Dim checkKeyword = Function(idx As Integer, ByRef lngth As Integer, word As String)
Dim l = word.Length
If idx + (l - 1) < text.Length Then
If text.Substring(idx, l).ToUpper = word Then
lngth = l
Return True
End If
End If
Return False
End Function
Dim c = text(index) analyze the current character
If Char.IsWhiteSpace(c) Then if whitespace, find consecutive whitespace
length = 1
Dim start = index
Do
start += 1
If Char.IsWhiteSpace(text(start)) Then
length += 1
Else
Exit Do
End If
Loop While start < text.Length
Return TokenKind.Whitespace
ElseIf Char.IsNumber(c) if numeric, find rest of numeric value
Dim start = index
length = 1
If index > 0 AndAlso text(index - 1) = "-"c Then check for preceeding negative
length += 1
index -= 1
End If
Do While start < text.Length - 1
start += 1
If Char.IsNumber(text(start)) Then
length += 1
ElseIf Char.ToUpper(text(start)) = "E"c check for exponent
length += 1
If text(start + 1) = "+"c OrElse text(start + 1) = "-"c Then
length += 1
End If
ElseIf text(start) = "."c check for decimal
length += 1
Else
Exit Do
End If
Loop
Return TokenKind.Number
Else c is not whitespace or numeric, analyze the character
Select Case c
Case "{"c starts an object
length = 1
Return TokenKind.OpenObject
Case "}"c ends an object
length = 1
Return TokenKind.CloseObject
Case "["c starts an array
length = 1
Return TokenKind.OpenArray
Case "]"c ends an array
length = 1
Return TokenKind.CloseArray
Case """"c starts a string, find rest of string
length = 1
Dim start = index
Do While start < text.Length
start += 1
length += 1
Select Case text(start)
Case "\"c
start += 1
length += 1
Case """"c
Exit Do
End Select
Loop
Return TokenKind.Text
Case ":"c marks an object property key/value pair seperator
length = 1
Return TokenKind.DelimitProperty
Case ","c marks an array value delimiter, or an object key/value pair delimiter.
length = 1
Return TokenKind.DelimitEntry
Case "t"c, "T"c should indicate a boolean value of "true"
If checkKeyword(index, length, "TRUE") Then Return TokenKind.Bool
Case "f"c, "F"c should inidicate a boolean value of "false"
If checkKeyword(index, length, "FALSE") Then Return TokenKind.Bool
Case "n"c, "N"c should indicate a "null" value
If checkKeyword(index, length, "NULL") Then Return TokenKind.Null
End Select
End If
Return TokenKind.UNKNOWN should only occur with invalid JSON text
End Function
<summary>
Performs lexing of the source JSON string into tokens for parsing.
</summary>
<param name="text">The raw JSON text string to lex.</param>
<returns>A List of Token instances lexed from the JSON string.</returns>
Private Shared Function Lex(text As String) As List(Of Token)
Dim tokens As New List(Of Token)
Dim index As Integer
While index < text.Length
Dim l As Integer = 0
Try
Dim k = FindToken(text, index, l)
If k = TokenKind.UNKNOWN Then Throw New LexerException("Unknown token encountered.", text, index, tokens.LastOrDefault.Kind)
If l = 0 Then Throw New LexerException("Zero-length token encountered.", text, index, tokens.LastOrDefault.Kind)
If Not k = TokenKind.Whitespace Then
tokens.Add(New Token(index, l, k))
End If
Catch ex As Exception
Throw New LexerException("Unexpected lexer error, see inner exception for details. This usually indicates invalid JSON text.", text, index, tokens.LastOrDefault.Kind, ex)
End Try
index += l
End While
Return tokens
End Function
<summary>
Creates a Boolean JsonValue for the specified text.
</summary>
<param name="text">The text from which the JsonValue is created.</param>
<returns>A Boolean JsonValue, if the text can be parsed as a Boolean value, otherwise a String JsonValue.</returns>
Private Shared Function MakeJsonBoolValue(text As String) As JsonValue
Dim textBool As Boolean
If Boolean.TryParse(text, textBool) Then
Return New JsonValue(textBool)
End If
Return New JsonValue(text)
End Function
<summary>
Creates a Number JsonValue for the specified text, setting the extended type as appropriate.
</summary>
<param name="text">The text from which the JsonValue is created.</param>
<returns>A Number JsonValue with the appropriate extended data type, if the text can be parsed as a numeric value, otherwise a String JsonValue.</returns>
Private Shared Function MakeJsonNumberValue(text As String) As JsonValue
Dim numberInt As Integer
If Integer.TryParse(text, numberInt) Then
Return New JsonValue(numberInt)
End If
Dim numberDec As Decimal
If Decimal.TryParse(text, numberDec) Then
Return New JsonValue(numberDec)
End If
Return New JsonValue(text)
End Function
<summary>
Creates a String JsonValue for the specified text, setting the Date extended type if applicable.
</summary>
<param name="text">The text from which the JsonValue is created.</param>
<returns>A String JsonValue with the appropriate extended data type.</returns>
Private Shared Function MakeJsonStringValue(text As String) As JsonValue
Dim dt As Date
If Date.TryParse(text, dt) Then
Return New JsonValue(dt)
End If
Return New JsonValue(text)
End Function
<summary>
Gets the unescaped value of a string containing JSON escape codes.
</summary>
<param name="token">The token containing the position and length within the source JSON text.</param>
<param name="text">The source JSON text containing the string to unescape.</param>
<returns>The unescaped value of a string containing JSON escape codes.</returns>
Private Shared Function UnescapeString(token As Token, text As String) As String
If token.Kind = TokenKind.Text Then
Dim substring = text.Substring(token.Index + 1, token.Length - 2)
Dim sb As New Text.StringBuilder
For i = 0 To substring.Length - 1
Dim c = substring(i)
If c = "\"c Then
If i < substring.Length - 1 Then
i += 1
Dim ec = substring(i)
Select Case ec
Case "\"c, "/"c, """"c
sb.Append(ec)
Case "b"c
If sb.Length > 0 Then sb.Length -= 1
Case "f"c
sb.Append(ControlChars.FormFeed)
Case "n"c
sb.Append(ControlChars.NewLine)
Case "r"c
sb.Append(ControlChars.Cr)
Case "t"c
sb.Append(ControlChars.Tab)
Case "u"c
If i + 4 < substring.Length Then
Dim charval As Int16
If Int16.TryParse(substring.Substring(i + 1, 4), Globalization.NumberStyles.HexNumber,
Globalization.CultureInfo.CurrentCulture.NumberFormat, charval) Then
sb.Append(ChrW(charval))
i += 4
End If
End If
End Select
End If
Else
sb.Append(c)
End If
Next
Return sb.ToString
End If
Return Nothing
End Function
Private Class JEntity
Public Property Kind As TokenKind
Public Property Value As Object
Public Sub New(ek As TokenKind, v As Object)
Kind = ek
Value = v
End Sub
End Class
Private Structure Token
Public Index As Integer
Public Length As Integer
Public Kind As TokenKind
Public Sub New(i As Integer, l As Integer, k As TokenKind)
Index = i
Length = l
Kind = k
End Sub
End Structure
<summary>
Indicates the type of token identified within a JSON string.
</summary>
Public Enum TokenKind
UNKNOWN
Null null
Text ""
Bool true/false
Number 123
OpenObject {
CloseObject }
OpenArray [
CloseArray ]
DelimitProperty :
DelimitEntry ,
Whitespace
End Enum
<summary>
This exception occurs when an error is encountered during lexing of a JSON string.
</summary>
Public Class LexerException
Inherits Exception
<summary>
The raw JSON text being lexed.
</summary>
Public ReadOnly JsonText As String
<summary>
The position within the string at which the error occured.
</summary>
Public ReadOnly Index As Integer
<summary>
The type of the last token successfully discovered.
</summary>
Public ReadOnly LastGoodTokenKind As TokenKind
Protected Friend Sub New(message As String, text As String, idx As Integer, k As TokenKind)
Me.New(message, text, idx, k, Nothing)
End Sub
Protected Friend Sub New(message As String, text As String, idx As Integer, k As TokenKind, inner As Exception)
MyBase.New(message, inner)
JsonText = text
Index = idx
LastGoodTokenKind = k
End Sub
End Class
<summary>
This exception occurs when an error is encountered during parsing of tokens, after lexing has occured.
</summary>
Public Class ParsingException
Inherits Exception
<summary>
The index of the token being parsed when the error occured.
</summary>
Public ReadOnly TokenIndex As Integer
<summary>
The type of the token being parsed when the error occured.
</summary>
Public ReadOnly TokenKind As TokenKind
<summary>
The type of the last token successfully parsed.
</summary>
Public ReadOnly PreviousTokenKind As TokenKind
Public Sub New(idx As Integer, k As TokenKind, pk As TokenKind, inner As Exception)
MyBase.New("An error occured while parsing the text, see inner exception for details. This indicates an error in the JSON document structure.", inner)
TokenIndex = idx
TokenKind = k
PreviousTokenKind = pk
End Sub
End Class
End Class
End Namespace
So thats the "beta" implementation of the library. There may still be some room for improvements, or bugs to squash, but so far Im pretty satisfied with the results of the tests Ive performed. Ill continue to add functionality and refine whats there, but I wanted to get this out here in the hopes of getting test results for various usage scenarios before putting it anywhere else (like GitHub).
Reed Kimble - "When you do things right, people wont be sure youve done anything at all"
Continue reading...
I believe the common standard for handling JSON in .Net is to use Newtonsoft.JSON, and it is great at what it does. When you know the object model which the JSON represents and have (or can easily construct) the POCO for that model, then it is really fast and easy to deserialize the JSON into a graph of object instances from the model, or query the model via LINQ.
But when you dont know what the model looks like, or dont have (or dont want to create) the plain-old-class-objects representing the model, then Newtonsoft.JSON can be a bit of a pain. Examining the hierarchy of the object model parsed by Newtonsoft.JSON is not intuitive and does not explicitly follow the JSON document format. These arent problems or criticisms; Netwtonsoft.JSON was simply not built for this purpose.
To address this, Ive built a small library Im calling "JsonCore" which follows the JSON document structure and provides simplified parsing of JSON document text into JsonValue objects.
A JsonValue object is the base object type of a JSON data element. It can represent one of a String, Number, Object, Array, Boolean, or Null as per the JSON data specification. A JsonValue instance contains IsDataType() methods which can be called to interrogate the instance and determine the JSON data element it represents. There are also associated AsDataType() methods to cast the JsonValue instance into the specific type, if the JsonValue instance represents an instance of the related JSON data element (if the data type does not match the return type of the method called, an InvalidOperationException will occur).
A JsonValue with the data type Array can be cast into a JsonArray instance. The JsonArray object will provide access to the collection of JsonValue objects contained within the JsonArray.
A JsonValue with the data type Object can be cast into a JsonObject instance. The JsonObject object will provide access to the named properties contained within the JsonObject. The object can also be enumerated as a collection of key-value-pairs of string and JsonValue.
This design makes it very easy to parse a JSON string and then analyze the resulting document hierarchy. The debug display in Visual Studio is able to provide a clean and structured view into the parsed object model. The resulting collection of JsonValue objects is also easy to navigate and consume with code.
Ive only implemented the consumption of JSON so far, not its generation, but this was the hard part; generating JSON is much easier and Ill probably add that functionality to the library in the future. Right now I want to concentrate on the parsing to ensure that the specification has been accurately followed and that there are no bugs in the implementation. This iteration of the code has executed quickly and accurately against three separate sources that Ive tested. I believe it is correct and bug-free, but you never know... Id love to get some feedback on how this performs for other JSON sources, and usability suggestions are welcome as well.
The library is presented in two code files; one to contain the JsonValue object definitions and one to contain the JsonParser implementation. First the object definitions. This code looks long, but it is mostly boilerplate. The objects themselves are actually quite simple.
Option Strict On
Namespace JsonCore
<summary>
Represents a JsonValue of type Array; one of the two fundemental JSON structures (Object and Array).
</summary>
Public Class JsonArray
Inherits JsonValue
Implements IEnumerable(Of JsonValue)
Public Shared Function FromJsonValue(value As JsonValue) As JsonArray
If value.IsArray Then Return CType(value, JsonArray)
Dim result = New JsonArray()
result.items.Add(value)
Return result
End Function
Private items As List(Of JsonValue)
<summary>
Creates a new, empty JsonArray instance.
</summary>
Public Sub New()
MyBase.New(JsonDataType.Array, Nothing)
items = DirectCast(_Value, List(Of JsonValue))
End Sub
<summary>
Gets the number of JsonValue objects contianed within this JsonArray.
</summary>
<returns>An Integer representing the number of JsonValues in the array.</returns>
Public ReadOnly Property Count As Integer
Get
Return items.Count
End Get
End Property
<summary>
Gets or sets the JsonValue at the specified index in this JsonArray.
</summary>
<param name="index">The index of the JsonValue to get or set.</param>
<returns>The JsonValue at the specified index.</returns>
Default Public Property Item(index As Integer) As JsonValue
Get
Return items(index)
End Get
Set(value As JsonValue)
items(index) = value
End Set
End Property
<summary>
Adds a new JsonValue instance to the end of this JsonArray.
</summary>
<param name="value">The JsonValue to append to the end of the array.</param>
Public Sub Push(value As JsonValue)
items.Add(value)
End Sub
<summary>
Gets an enumerator of JsonValues contained within this JsonArray.
</summary>
<returns>An enumerator of JsonValues contained within this JsonArray.</returns>
Public Function GetEnumerator() As IEnumerator(Of JsonValue) Implements IEnumerable(Of JsonValue).GetEnumerator
Return items.GetEnumerator
End Function
Private Function IEnumerable_GetEnumerator() As IEnumerator Implements IEnumerable.GetEnumerator
Return GetEnumerator()
End Function
End Class
<summary>
Represents a JsonValue of type Object; one of the two fundemental JSON structures (Object and Array).
</summary>
Public Class JsonObject
Inherits JsonValue
Implements IEnumerable(Of KeyValuePair(Of String, JsonValue))
Private properties As Dictionary(Of String, JsonValue)
<summary>
Gets or sets a named property value on this JsonObject.
</summary>
<param name="name">The name of the property to get or set.</param>
<returns>A JsonValue representing the value of the specified property.</returns>
Default Public Property [Property](name As String) As JsonValue
Get
Return properties(name)
End Get
Set(value As JsonValue)
properties(name) = value
End Set
End Property
<summary>
Gets the number of properties on this JsonObject.
</summary>
<returns>An Integer representing the number of properties on this JsonObject.</returns>
Public ReadOnly Property PropertyCount As Integer
Get
Return properties.Count
End Get
End Property
<summary>
Gets the property name at the specified index, in the order the properties were discovered.
</summary>
<param name="index">The index of the property name to get.</param>
<returns>A String representing the property name at the specified index.</returns>
Public ReadOnly Property PropertyName(index As Integer) As String
Get
Return properties.Keys.ElementAt(index)
End Get
End Property
<summary>
Gets the collection of property names for this JsonObject.
</summary>
<returns>An IEnumerable of Strings containing the property names defined in this JsonObject.</returns>
Public ReadOnly Property PropertyNames As IEnumerable(Of String)
Get
Return properties.Keys
End Get
End Property
<summary>
Creates a new JsonObject instance with no defined properties.
</summary>
Public Sub New()
MyBase.New(JsonDataType.Object, Nothing)
properties = DirectCast(_Value, Dictionary(Of String, JsonValue))
End Sub
<summary>
Gets an enumerator of key/value pairs of Strings and JsonValues, representing the properties of this JsonObject.
</summary>
<returns>An enumerator of key/value pairs of Strings and JsonValues representing the properties of this JsonObject.</returns>
Public Function GetEnumerator() As IEnumerator(Of KeyValuePair(Of String, JsonValue)) Implements IEnumerable(Of KeyValuePair(Of String, JsonValue)).GetEnumerator
Return properties.GetEnumerator
End Function
Private Function IEnumerable_GetEnumerator() As IEnumerator Implements IEnumerable.GetEnumerator
Return GetEnumerator()
End Function
End Class
<summary>
Represents a value in a JSON document of a particular data type. Valid JSON data types are String, Number, Object,
Array, Boolean, and Null. Strings may have an extended type of Date, and Numbers will have an extended type of
either Integer or Decimal.
</summary>
Public Class JsonValue
Protected _Value As Object
<summary>
Gets a value from the JsonDataType enum representing the JSON data type for this value.
</summary>
<returns></returns>
Public ReadOnly Property ValueType As JsonDataType
<summary>
Gets the extended data type (Date, Integer, or Decimal) for the value, if applicable.
</summary>
<returns></returns>
Public ReadOnly Property ExtendedType As JsonExtendedType
<summary>
Creates a new Null JsonValue instance.
</summary>
Public Sub New()
Me.New(JsonDataType.Null, Nothing)
End Sub
<summary>
Creates a new Number JsonValue instance with the extended type Decimal.
</summary>
<param name="value">The Decimal value assigned to this JsonValue instance.</param>
Public Sub New(value As Decimal)
Me.New(JsonExtendedType.Decimal, value)
End Sub
<summary>
Creates a new Number JsonValue instance with the extended type Integer.
</summary>
<param name="value">The Integer value assigned to this JsonValue instance.</param>
Public Sub New(value As Integer)
Me.New(JsonExtendedType.Integer, value)
End Sub
<summary>
Creates a new String JsonValue instance.
</summary>
<param name="value">The String value assigned to this JsonValue instance.</param>
Public Sub New(value As String)
Me.New(JsonDataType.String, value)
End Sub
<summary>
Creates a new String JsonValue instance with the extended type Date.
</summary>
<param name="value">The Date value assigned to this JsonValue instance.</param>
Public Sub New(value As Date)
Me.New(JsonExtendedType.Date, value)
End Sub
<summary>
Creates a new Boolean JsonValue instance.
</summary>
<param name="value">The Boolean value assigned to this JsonValue instance.</param>
Public Sub New(value As Boolean)
Me.New(JsonDataType.Boolean, value)
End Sub
Protected Sub New(type As JsonDataType, value As Object)
ValueType = type
Initialize(value)
End Sub
Protected Sub New(type As JsonExtendedType, value As Object)
Select Case type
Case JsonExtendedType.Date, JsonExtendedType.None
ValueType = JsonDataType.String
Case JsonExtendedType.Decimal, JsonExtendedType.Integer
ValueType = JsonDataType.Number
End Select
ExtendedType = type
Initialize(value)
End Sub
Protected Sub Initialize(value As Object)
Select Case ValueType
Case JsonDataType.Array
_Value = New List(Of JsonValue)
If TypeOf value Is IEnumerable(Of JsonValue) Then
DirectCast(_Value, List(Of JsonValue)).AddRange(DirectCast(value, IEnumerable(Of JsonValue)))
ElseIf TypeOf value Is JsonValue
DirectCast(_Value, List(Of JsonValue)).Add(DirectCast(value, JsonValue))
End If
Case JsonDataType.Object
If TypeOf value Is IDictionary(Of String, JsonValue) Then
_Value = New Dictionary(Of String, JsonValue)(DirectCast(value, IDictionary(Of String, JsonValue)))
Else
_Value = New Dictionary(Of String, JsonValue)
If value?.GetType.IsArray Then
For Each entry In CType(value, Array)
If entry.GetType.IsArray Then
Dim values = CType(entry, Array)
If values.Length = 2 Then
If TypeOf values.GetValue(0) Is String Then
If TypeOf values.GetValue(1) Is JsonValue Then
DirectCast(_Value, Dictionary(Of String, JsonValue)).Add(DirectCast(values.GetValue(0), String), DirectCast(values.GetValue(1), JsonValue))
End If
End If
End If
End If
Next
End If
End If
Case JsonDataType.Boolean
_Value = CBool(value)
Case JsonDataType.Null
_Value = Nothing
Case JsonDataType.Number
Select Case ExtendedType
Case JsonExtendedType.Integer
_Value = CInt(value)
Case JsonExtendedType.Decimal
_Value = CDec(value)
End Select
Case JsonDataType.String
_Value = CStr(value)
Case Else
_Value = value
End Select
End Sub
<summary>
Gets this JsonValue instance as a JsonArray object, if the instance represents a JsonArray.
Otherwise throws an InvalidOperationException.
</summary>
<returns>A JsonArray instance representing this JsonValue.</returns>
Public Function AsArray() As JsonArray
If ValueType = JsonDataType.Array Then Return DirectCast(Me, JsonArray)
Throw New InvalidOperationException("JsonDataType is not Array.")
End Function
<summary>
Gets this JsonValue instance as a Boolean object, if the instance represents a Boolean.
Otherwise throws an InvalidOperationException.
</summary>
<returns>A Boolean instance representing this JsonValue.</returns>
Public Function AsBoolean() As Boolean
If ValueType = JsonDataType.Boolean Then Return DirectCast(_Value, Boolean)
Throw New InvalidOperationException("JsonDataType is not Boolean.")
End Function
<summary>
Gets this JsonValue instance as a Date object, if the instance represents a Date.
Otherwise throws an InvalidOperationException.
</summary>
<returns>A Date instance representing this JsonValue.</returns>
Public Function AsDate() As Date
If ExtendedType = JsonExtendedType.Date Then Return DirectCast(_Value, Date)
Throw New InvalidOperationException("JsonExtendedType is not Date.")
End Function
<summary>
Gets this JsonValue instance as a Decimal object, if the instance represents a Decimal.
Otherwise throws an InvalidOperationException.
</summary>
<returns>A Decimal instance representing this JsonValue.</returns>
Public Function AsDecimal() As Decimal
If ExtendedType = JsonExtendedType.Decimal Then Return DirectCast(_Value, Decimal)
Throw New InvalidOperationException("JsonExtendedType is not Decimal.")
End Function
<summary>
Gets this JsonValue instance as an Integer object, if the instance represents an Integer.
Otherwise throws an InvalidOperationException.
</summary>
<returns>An Integer instance representing this JsonValue.</returns>
Public Function AsInteger() As Integer
If ExtendedType = JsonExtendedType.Integer Then Return DirectCast(_Value, Integer)
Throw New InvalidOperationException("JsonExtendedType is not Integer.")
End Function
<summary>
Gets this JsonValue instance as a JsonObject object, if the instance represents a JsonObject.
Otherwise throws an InvalidOperationException.
</summary>
<returns>A JsonObject instance representing this JsonValue.</returns>
Public Function AsObject() As JsonObject
If ValueType = JsonDataType.Object Then Return DirectCast(Me, JsonObject)
Throw New InvalidOperationException("JsonDataType is not Object.")
End Function
<summary>
Gets this JsonValue instance as a String object, if the instance represents a String.
Otherwise throws an InvalidOperationException.
</summary>
<returns>A String instance representing this JsonValue.</returns>
Public Function AsString() As String
If ValueType = JsonDataType.String Then Return DirectCast(_Value, String)
Throw New InvalidOperationException("JsonDataType is not String.")
End Function
Public Overrides Function Equals(obj As Object) As Boolean
If TypeOf obj Is JsonValue Then
Dim other = DirectCast(obj, JsonValue)
If ValueType = other.ValueType Then
Return Comparer.Default.Compare(_Value, other._Value) = 0
End If
End If
Return False
End Function
<summary>
Gets a value indicating whether or not this JsonValue instance represents a JsonArray.
</summary>
<returns>True if this JsonValue is a JsonArray, otherwise false.</returns>
Public Function IsArray() As Boolean
Return ValueType = JsonDataType.Array
End Function
<summary>
Gets a value indicating whether or not this JsonValue instance represents a Boolean.
</summary>
<returns>True if this JsonValue is a Boolean, otherwise false.</returns>
Public Function IsBoolean() As Boolean
Return ValueType = JsonDataType.Boolean
End Function
<summary>
Gets a value indicating whether or not this JsonValue instance represents a Date.
</summary>
<returns>True if this JsonValue is a Date, otherwise false.</returns>
Public Function IsDate() As Boolean
Return ExtendedType = JsonExtendedType.Date
End Function
<summary>
Gets a value indicating whether or not this JsonValue instance represents a Decimal.
</summary>
<returns>True if this JsonValue is a Decimal, otherwise false.</returns>
Public Function IsDecimal() As Boolean
Return ExtendedType = JsonExtendedType.Decimal
End Function
<summary>
Gets a value indicating whether or not this JsonValue instance represents an Integer.
</summary>
<returns>True if this JsonValue is an Integer, otherwise false.</returns>
Public Function IsInteger() As Boolean
Return ExtendedType = JsonExtendedType.Integer
End Function
<summary>
Gets a value indicating whether or not this JsonValue instance represents Null.
</summary>
<returns>True if this JsonValue is Null, otherwise false.</returns>
Public Function IsNull() As Boolean
Return ValueType = JsonDataType.Null
End Function
<summary>
Gets a value indicating whether or not this JsonValue instance represents a JsonObject.
</summary>
<returns>True if this JsonValue is a JsonObject, otherwise false.</returns>
Public Function IsObject() As Boolean
Return ValueType = JsonDataType.Object
End Function
<summary>
Gets a value indicating whether or not this JsonValue instance represents a String.
</summary>
<returns>True if this JsonValue is a String, otherwise false.</returns>
Public Function IsString() As Boolean
Return ValueType = JsonDataType.String
End Function
Public Overloads Function ReferenceEquals(other As Object) As Boolean
Return Me Is other
End Function
Public Overrides Function ToString() As String
If _Value Is Nothing Then Return String.Empty
Return _Value.ToString
End Function
End Class
<summary>
Specifies the JSON data type of a JsonValue
</summary>
Public Enum JsonDataType
Null
[String]
Number
[Boolean]
Array
[Object]
End Enum
<summary>
Specifies the extended data type of a JSON String or Number.
</summary>
Public Enum JsonExtendedType
None
[Date]
[Decimal]
[Integer]
End Enum
End Namespace
Next comes the parser implementation and related code files. This is the complex part. The API is fully commented and there are a few code comments sprinkled throughout to help describe the purpose of the various code blocks, so hopefully it isnt too terribly difficult to follow along and understand how the parsing engine works. The JSON specification is pretty simple so it doesnt require a very complex parser. I tried to create a robust parsing engine... hopefully it is not overcomplicated.
Option Strict On
Namespace JsonCore
<summary>
Parses a JSON document string into a collection of JsonValue objects.
</summary>
Public Class JsonParser
<summary>
Creates a collection of JsonValue objects from the given JSON text string.
</summary>
<param name="text">The JSON string to parse into a JsonValue collection.</param>
<returns>An IEnumerable instance of JsonValue objects parsed from the supplied string.</returns>
Public Shared Function Parse(text As String) As IEnumerable(Of JsonValue)
Dim result As New List(Of JsonValue) holds the final, parsed JsonValue instances to return
Dim entities As New Stack(Of JEntity) stores the in-process entities being parsed
Dim tokens As List(Of Token) = Lex(text) hold the list of tokens returned from the lexer
Dim processClosure = Sub(value As Object) define helper to generate JsonValue instances from parsed entities
Dim topEntity = entities.Peek
Select Case topEntity.Kind
Case TokenKind.OpenArray
If TypeOf value Is JsonValue Then
DirectCast(topEntity.Value, JsonArray).Push(DirectCast(value, JsonValue))
Else
DirectCast(topEntity.Value, JsonArray).Push(MakeJsonStringValue(DirectCast(value, String)))
End If
Case TokenKind.DelimitEntry
entities.Pop()
topEntity = entities.Peek
If topEntity.Kind = TokenKind.OpenArray Then
If TypeOf value Is JsonValue Then
DirectCast(topEntity.Value, JsonArray).Push(DirectCast(value, JsonValue))
Else
DirectCast(topEntity.Value, JsonArray).Push(MakeJsonStringValue(DirectCast(value, String)))
End If
ElseIf topEntity.Kind = TokenKind.OpenObject
entities.Push(New JEntity(TokenKind.Text, value))
End If
Case TokenKind.OpenObject
entities.Push(New JEntity(TokenKind.Text, value))
Case TokenKind.DelimitProperty
entities.Pop()
Dim key = entities.Pop.Value.ToString
topEntity = entities.Peek
If TypeOf value Is JsonValue Then
DirectCast(topEntity.Value, JsonObject).Property(key) = DirectCast(value, JsonValue)
Else
DirectCast(topEntity.Value, JsonObject).Property(key) = MakeJsonStringValue(DirectCast(value, String))
End If
End Select
End Sub
Dim index As Integer index used to loop over token collection
Try
For index = 0 To tokens.Count - 1 Parse the token list into entities
Dim thisToken = tokens(index)
Select Case thisToken.Kind
Case TokenKind.OpenArray
entities.Push(New JEntity(TokenKind.OpenArray, New JsonArray))
Case TokenKind.OpenObject
entities.Push(New JEntity(TokenKind.OpenObject, New JsonObject))
Case TokenKind.CloseArray
Dim closedArray = DirectCast(entities.Pop.Value, JsonArray)
If entities.Count > 0 Then
processClosure(closedArray)
Else
result.Add(closedArray)
End If
Case TokenKind.CloseObject
Dim closedObject = DirectCast(entities.Pop.Value, JsonObject)
If entities.Count > 0 Then
processClosure(closedObject)
Else
result.Add(closedObject)
End If
Case TokenKind.Text
processClosure(UnescapeString(thisToken, text))
Case TokenKind.Number
processClosure(MakeJsonNumberValue(text.Substring(thisToken.Index, thisToken.Length)))
Case TokenKind.Bool
processClosure(MakeJsonBoolValue(text.Substring(thisToken.Index, thisToken.Length)))
Case TokenKind.Null
processClosure(New JsonValue())
Case TokenKind.DelimitProperty
entities.Push(New JEntity(TokenKind.DelimitProperty, Nothing))
Case TokenKind.DelimitEntry
entities.Push(New JEntity(TokenKind.DelimitEntry, Nothing))
End Select
Next
Catch ex As Exception
Throw New ParsingException(index, tokens(index).Kind, tokens.ElementAtOrDefault(index - 1).Kind, ex)
End Try
Return result.AsEnumerable
End Function
<summary>
Aids the lexer in finding each tokens start index and length within the source JSON string.
</summary>
<param name="text">The raw JSON text string.</param>
<param name="index">The current index being analyzed within the raw string.</param>
<param name="length">(OUT) Returns the length of the discovered token within the raw string.</param>
<returns>A value from the TokenKind enum representing the type of token discovered.</returns>
Private Shared Function FindToken(text As String, ByRef index As Integer, ByRef length As Integer) As TokenKind
helper for matching TRUE/FALSE/NULL values
Dim checkKeyword = Function(idx As Integer, ByRef lngth As Integer, word As String)
Dim l = word.Length
If idx + (l - 1) < text.Length Then
If text.Substring(idx, l).ToUpper = word Then
lngth = l
Return True
End If
End If
Return False
End Function
Dim c = text(index) analyze the current character
If Char.IsWhiteSpace(c) Then if whitespace, find consecutive whitespace
length = 1
Dim start = index
Do
start += 1
If Char.IsWhiteSpace(text(start)) Then
length += 1
Else
Exit Do
End If
Loop While start < text.Length
Return TokenKind.Whitespace
ElseIf Char.IsNumber(c) if numeric, find rest of numeric value
Dim start = index
length = 1
If index > 0 AndAlso text(index - 1) = "-"c Then check for preceeding negative
length += 1
index -= 1
End If
Do While start < text.Length - 1
start += 1
If Char.IsNumber(text(start)) Then
length += 1
ElseIf Char.ToUpper(text(start)) = "E"c check for exponent
length += 1
If text(start + 1) = "+"c OrElse text(start + 1) = "-"c Then
length += 1
End If
ElseIf text(start) = "."c check for decimal
length += 1
Else
Exit Do
End If
Loop
Return TokenKind.Number
Else c is not whitespace or numeric, analyze the character
Select Case c
Case "{"c starts an object
length = 1
Return TokenKind.OpenObject
Case "}"c ends an object
length = 1
Return TokenKind.CloseObject
Case "["c starts an array
length = 1
Return TokenKind.OpenArray
Case "]"c ends an array
length = 1
Return TokenKind.CloseArray
Case """"c starts a string, find rest of string
length = 1
Dim start = index
Do While start < text.Length
start += 1
length += 1
Select Case text(start)
Case "\"c
start += 1
length += 1
Case """"c
Exit Do
End Select
Loop
Return TokenKind.Text
Case ":"c marks an object property key/value pair seperator
length = 1
Return TokenKind.DelimitProperty
Case ","c marks an array value delimiter, or an object key/value pair delimiter.
length = 1
Return TokenKind.DelimitEntry
Case "t"c, "T"c should indicate a boolean value of "true"
If checkKeyword(index, length, "TRUE") Then Return TokenKind.Bool
Case "f"c, "F"c should inidicate a boolean value of "false"
If checkKeyword(index, length, "FALSE") Then Return TokenKind.Bool
Case "n"c, "N"c should indicate a "null" value
If checkKeyword(index, length, "NULL") Then Return TokenKind.Null
End Select
End If
Return TokenKind.UNKNOWN should only occur with invalid JSON text
End Function
<summary>
Performs lexing of the source JSON string into tokens for parsing.
</summary>
<param name="text">The raw JSON text string to lex.</param>
<returns>A List of Token instances lexed from the JSON string.</returns>
Private Shared Function Lex(text As String) As List(Of Token)
Dim tokens As New List(Of Token)
Dim index As Integer
While index < text.Length
Dim l As Integer = 0
Try
Dim k = FindToken(text, index, l)
If k = TokenKind.UNKNOWN Then Throw New LexerException("Unknown token encountered.", text, index, tokens.LastOrDefault.Kind)
If l = 0 Then Throw New LexerException("Zero-length token encountered.", text, index, tokens.LastOrDefault.Kind)
If Not k = TokenKind.Whitespace Then
tokens.Add(New Token(index, l, k))
End If
Catch ex As Exception
Throw New LexerException("Unexpected lexer error, see inner exception for details. This usually indicates invalid JSON text.", text, index, tokens.LastOrDefault.Kind, ex)
End Try
index += l
End While
Return tokens
End Function
<summary>
Creates a Boolean JsonValue for the specified text.
</summary>
<param name="text">The text from which the JsonValue is created.</param>
<returns>A Boolean JsonValue, if the text can be parsed as a Boolean value, otherwise a String JsonValue.</returns>
Private Shared Function MakeJsonBoolValue(text As String) As JsonValue
Dim textBool As Boolean
If Boolean.TryParse(text, textBool) Then
Return New JsonValue(textBool)
End If
Return New JsonValue(text)
End Function
<summary>
Creates a Number JsonValue for the specified text, setting the extended type as appropriate.
</summary>
<param name="text">The text from which the JsonValue is created.</param>
<returns>A Number JsonValue with the appropriate extended data type, if the text can be parsed as a numeric value, otherwise a String JsonValue.</returns>
Private Shared Function MakeJsonNumberValue(text As String) As JsonValue
Dim numberInt As Integer
If Integer.TryParse(text, numberInt) Then
Return New JsonValue(numberInt)
End If
Dim numberDec As Decimal
If Decimal.TryParse(text, numberDec) Then
Return New JsonValue(numberDec)
End If
Return New JsonValue(text)
End Function
<summary>
Creates a String JsonValue for the specified text, setting the Date extended type if applicable.
</summary>
<param name="text">The text from which the JsonValue is created.</param>
<returns>A String JsonValue with the appropriate extended data type.</returns>
Private Shared Function MakeJsonStringValue(text As String) As JsonValue
Dim dt As Date
If Date.TryParse(text, dt) Then
Return New JsonValue(dt)
End If
Return New JsonValue(text)
End Function
<summary>
Gets the unescaped value of a string containing JSON escape codes.
</summary>
<param name="token">The token containing the position and length within the source JSON text.</param>
<param name="text">The source JSON text containing the string to unescape.</param>
<returns>The unescaped value of a string containing JSON escape codes.</returns>
Private Shared Function UnescapeString(token As Token, text As String) As String
If token.Kind = TokenKind.Text Then
Dim substring = text.Substring(token.Index + 1, token.Length - 2)
Dim sb As New Text.StringBuilder
For i = 0 To substring.Length - 1
Dim c = substring(i)
If c = "\"c Then
If i < substring.Length - 1 Then
i += 1
Dim ec = substring(i)
Select Case ec
Case "\"c, "/"c, """"c
sb.Append(ec)
Case "b"c
If sb.Length > 0 Then sb.Length -= 1
Case "f"c
sb.Append(ControlChars.FormFeed)
Case "n"c
sb.Append(ControlChars.NewLine)
Case "r"c
sb.Append(ControlChars.Cr)
Case "t"c
sb.Append(ControlChars.Tab)
Case "u"c
If i + 4 < substring.Length Then
Dim charval As Int16
If Int16.TryParse(substring.Substring(i + 1, 4), Globalization.NumberStyles.HexNumber,
Globalization.CultureInfo.CurrentCulture.NumberFormat, charval) Then
sb.Append(ChrW(charval))
i += 4
End If
End If
End Select
End If
Else
sb.Append(c)
End If
Next
Return sb.ToString
End If
Return Nothing
End Function
Private Class JEntity
Public Property Kind As TokenKind
Public Property Value As Object
Public Sub New(ek As TokenKind, v As Object)
Kind = ek
Value = v
End Sub
End Class
Private Structure Token
Public Index As Integer
Public Length As Integer
Public Kind As TokenKind
Public Sub New(i As Integer, l As Integer, k As TokenKind)
Index = i
Length = l
Kind = k
End Sub
End Structure
<summary>
Indicates the type of token identified within a JSON string.
</summary>
Public Enum TokenKind
UNKNOWN
Null null
Text ""
Bool true/false
Number 123
OpenObject {
CloseObject }
OpenArray [
CloseArray ]
DelimitProperty :
DelimitEntry ,
Whitespace
End Enum
<summary>
This exception occurs when an error is encountered during lexing of a JSON string.
</summary>
Public Class LexerException
Inherits Exception
<summary>
The raw JSON text being lexed.
</summary>
Public ReadOnly JsonText As String
<summary>
The position within the string at which the error occured.
</summary>
Public ReadOnly Index As Integer
<summary>
The type of the last token successfully discovered.
</summary>
Public ReadOnly LastGoodTokenKind As TokenKind
Protected Friend Sub New(message As String, text As String, idx As Integer, k As TokenKind)
Me.New(message, text, idx, k, Nothing)
End Sub
Protected Friend Sub New(message As String, text As String, idx As Integer, k As TokenKind, inner As Exception)
MyBase.New(message, inner)
JsonText = text
Index = idx
LastGoodTokenKind = k
End Sub
End Class
<summary>
This exception occurs when an error is encountered during parsing of tokens, after lexing has occured.
</summary>
Public Class ParsingException
Inherits Exception
<summary>
The index of the token being parsed when the error occured.
</summary>
Public ReadOnly TokenIndex As Integer
<summary>
The type of the token being parsed when the error occured.
</summary>
Public ReadOnly TokenKind As TokenKind
<summary>
The type of the last token successfully parsed.
</summary>
Public ReadOnly PreviousTokenKind As TokenKind
Public Sub New(idx As Integer, k As TokenKind, pk As TokenKind, inner As Exception)
MyBase.New("An error occured while parsing the text, see inner exception for details. This indicates an error in the JSON document structure.", inner)
TokenIndex = idx
TokenKind = k
PreviousTokenKind = pk
End Sub
End Class
End Class
End Namespace
So thats the "beta" implementation of the library. There may still be some room for improvements, or bugs to squash, but so far Im pretty satisfied with the results of the tests Ive performed. Ill continue to add functionality and refine whats there, but I wanted to get this out here in the hopes of getting test results for various usage scenarios before putting it anywhere else (like GitHub).
Reed Kimble - "When you do things right, people wont be sure youve done anything at all"
Continue reading...