[jira] Created: (MATH-312) RealVector interface could use some iterators (dense and sparse) and generic map() and collect() methods.

7 messages Options
Embed this post
Permalink
JIRA jira@apache.org

[jira] Created: (MATH-312) RealVector interface could use some iterators (dense and sparse) and generic map() and collect() methods.

Reply Threaded More More options
Print post
Permalink
RealVector interface could use some iterators (dense and sparse) and generic map() and collect() methods.
---------------------------------------------------------------------------------------------------------

                 Key: MATH-312
                 URL: https://issues.apache.org/jira/browse/MATH-312
             Project: Commons Math
          Issue Type: New Feature
    Affects Versions: 2.0
         Environment: all
            Reporter: Jake Mannix
             Fix For: 2.1


As discussed on the [math] list, there are other projects out there which would love to get a chance to standardize on using commons-math for things like linear algebra primitives, as it would build a common base to build upon.  But to do that, some well-known and used techniques for dealing with vectors, for one thing, are missing.  Most glaringly is the treatment of sparse vectors: giving no Iterator for non-default values means external clients lose the advantage of sparseness - only internal methods can skip around.  

Extending the RealVector interface with sparse (and dense) iterator methods would fix this:

{code}
  double getDefaultValue();
  Iterator<RealVector.Entry> iterator();
  Iterator<RealVector.Entry> nonDefaultIterator();
{code}

but there is another way to deal with vector data as well: instead of passing iterators around, and worrying about all the lovely ConcurrentModification and unsupported "remove" methods (which aren't the end of the world), we can instead expose generic map functions:

{code}
  RealVector map(UnivariateRealFunction f);
  RealVector mapToSelf(UnivariateRealFunction f);
{code}

where RealVector mapToSelf(UnivariateRealFunction), which applies the function to the vector's entries (checking whether the function preserves the default value up front allows it to chose between the sparse or dense iterator), and map just applies mapToSelf to a copy.

This doesn't exhaust all possible places where Iterators could be used helpfully (there's also combining two vectors together via a {code}map(BinaryRealFunction, RealVector other){code} which could be specialized nonlinear forms of addition or subtraction, and {code}double collect(UnivariateRealFunction, BinaryRealFunction){code} which uses the iterates over all of the entries, applying the first unary function to each entry, and then applying the binary function to combine this value with the previous accumulated value - with "pow(2)", and "+" as the two functions, you get L2 norm, with "abs()" and "+", you get L1 norm, etc...)


--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

JIRA jira@apache.org

[jira] Updated: (MATH-312) RealVector interface could use some iterators (dense and sparse) and generic map() and collect() methods.

Reply Threaded More More options
Print post
Permalink

     [ https://issues.apache.org/jira/browse/MATH-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jake Mannix updated MATH-312:
-----------------------------

    Attachment: MATH-312.patch

Initial patch - sorry about my formatting, I haven't got checkstyle quite playing nice enough to just reformat all the time correctly for me.

This patch has an AbstractRealVector which implements the iterators and map functions, as well as faster iterators in ArrayRealVector and OpenMapRealVector.

> RealVector interface could use some iterators (dense and sparse) and generic map() and collect() methods.
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: MATH-312
>                 URL: https://issues.apache.org/jira/browse/MATH-312
>             Project: Commons Math
>          Issue Type: New Feature
>    Affects Versions: 2.0
>         Environment: all
>            Reporter: Jake Mannix
>             Fix For: 2.1
>
>         Attachments: MATH-312.patch
>
>
> As discussed on the [math] list, there are other projects out there which would love to get a chance to standardize on using commons-math for things like linear algebra primitives, as it would build a common base to build upon.  But to do that, some well-known and used techniques for dealing with vectors, for one thing, are missing.  Most glaringly is the treatment of sparse vectors: giving no Iterator for non-default values means external clients lose the advantage of sparseness - only internal methods can skip around.  
> Extending the RealVector interface with sparse (and dense) iterator methods would fix this:
> {code}
>   double getDefaultValue();
>   Iterator<RealVector.Entry> iterator();
>   Iterator<RealVector.Entry> nonDefaultIterator();
> {code}
> but there is another way to deal with vector data as well: instead of passing iterators around, and worrying about all the lovely ConcurrentModification and unsupported "remove" methods (which aren't the end of the world), we can instead expose generic map functions:
> {code}
>   RealVector map(UnivariateRealFunction f);
>   RealVector mapToSelf(UnivariateRealFunction f);
> {code}
> where RealVector mapToSelf(UnivariateRealFunction), which applies the function to the vector's entries (checking whether the function preserves the default value up front allows it to chose between the sparse or dense iterator), and map just applies mapToSelf to a copy.
> This doesn't exhaust all possible places where Iterators could be used helpfully (there's also combining two vectors together via a {code}map(BinaryRealFunction, RealVector other){code} which could be specialized nonlinear forms of addition or subtraction, and {code}double collect(UnivariateRealFunction, BinaryRealFunction){code} which uses the iterates over all of the entries, applying the first unary function to each entry, and then applying the binary function to combine this value with the previous accumulated value - with "pow(2)", and "+" as the two functions, you get L2 norm, with "abs()" and "+", you get L1 norm, etc...)

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

JIRA jira@apache.org

[jira] Commented: (MATH-312) RealVector interface could use some iterators (dense and sparse) and generic map() and collect() methods.

Reply Threaded More More options
Print post
Permalink
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/MATH-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771317#action_12771317 ]

Jake Mannix commented on MATH-312:
----------------------------------

Oh yeah, patch also has a new unit test, which is nowhere near comprehensive, but does the basic sanity check.  All previous tests still pass, of course.

> RealVector interface could use some iterators (dense and sparse) and generic map() and collect() methods.
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: MATH-312
>                 URL: https://issues.apache.org/jira/browse/MATH-312
>             Project: Commons Math
>          Issue Type: New Feature
>    Affects Versions: 2.0
>         Environment: all
>            Reporter: Jake Mannix
>             Fix For: 2.1
>
>         Attachments: MATH-312.patch
>
>
> As discussed on the [math] list, there are other projects out there which would love to get a chance to standardize on using commons-math for things like linear algebra primitives, as it would build a common base to build upon.  But to do that, some well-known and used techniques for dealing with vectors, for one thing, are missing.  Most glaringly is the treatment of sparse vectors: giving no Iterator for non-default values means external clients lose the advantage of sparseness - only internal methods can skip around.  
> Extending the RealVector interface with sparse (and dense) iterator methods would fix this:
> {code}
>   double getDefaultValue();
>   Iterator<RealVector.Entry> iterator();
>   Iterator<RealVector.Entry> nonDefaultIterator();
> {code}
> but there is another way to deal with vector data as well: instead of passing iterators around, and worrying about all the lovely ConcurrentModification and unsupported "remove" methods (which aren't the end of the world), we can instead expose generic map functions:
> {code}
>   RealVector map(UnivariateRealFunction f);
>   RealVector mapToSelf(UnivariateRealFunction f);
> {code}
> where RealVector mapToSelf(UnivariateRealFunction), which applies the function to the vector's entries (checking whether the function preserves the default value up front allows it to chose between the sparse or dense iterator), and map just applies mapToSelf to a copy.
> This doesn't exhaust all possible places where Iterators could be used helpfully (there's also combining two vectors together via a {code}map(BinaryRealFunction, RealVector other){code} which could be specialized nonlinear forms of addition or subtraction, and {code}double collect(UnivariateRealFunction, BinaryRealFunction){code} which uses the iterates over all of the entries, applying the first unary function to each entry, and then applying the binary function to combine this value with the previous accumulated value - with "pow(2)", and "+" as the two functions, you get L2 norm, with "abs()" and "+", you get L1 norm, etc...)

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

JIRA jira@apache.org

[jira] Commented: (MATH-312) RealVector interface could use some iterators (dense and sparse) and generic map() and collect() methods.

Reply Threaded More More options
Print post
Permalink
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/MATH-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12772474#action_12772474 ]

Jake Mannix commented on MATH-312:
----------------------------------

This patch needs revision - the non-zero defaultValue() is not handled properly in OpenMapRealVector in this one.

> RealVector interface could use some iterators (dense and sparse) and generic map() and collect() methods.
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: MATH-312
>                 URL: https://issues.apache.org/jira/browse/MATH-312
>             Project: Commons Math
>          Issue Type: New Feature
>    Affects Versions: 2.0
>         Environment: all
>            Reporter: Jake Mannix
>             Fix For: 2.1
>
>         Attachments: MATH-312.patch
>
>
> As discussed on the [math] list, there are other projects out there which would love to get a chance to standardize on using commons-math for things like linear algebra primitives, as it would build a common base to build upon.  But to do that, some well-known and used techniques for dealing with vectors, for one thing, are missing.  Most glaringly is the treatment of sparse vectors: giving no Iterator for non-default values means external clients lose the advantage of sparseness - only internal methods can skip around.  
> Extending the RealVector interface with sparse (and dense) iterator methods would fix this:
> {code}
>   double getDefaultValue();
>   Iterator<RealVector.Entry> iterator();
>   Iterator<RealVector.Entry> nonDefaultIterator();
> {code}
> but there is another way to deal with vector data as well: instead of passing iterators around, and worrying about all the lovely ConcurrentModification and unsupported "remove" methods (which aren't the end of the world), we can instead expose generic map functions:
> {code}
>   RealVector map(UnivariateRealFunction f);
>   RealVector mapToSelf(UnivariateRealFunction f);
> {code}
> where RealVector mapToSelf(UnivariateRealFunction), which applies the function to the vector's entries (checking whether the function preserves the default value up front allows it to chose between the sparse or dense iterator), and map just applies mapToSelf to a copy.
> This doesn't exhaust all possible places where Iterators could be used helpfully (there's also combining two vectors together via a {code}map(BinaryRealFunction, RealVector other){code} which could be specialized nonlinear forms of addition or subtraction, and {code}double collect(UnivariateRealFunction, BinaryRealFunction){code} which uses the iterates over all of the entries, applying the first unary function to each entry, and then applying the binary function to combine this value with the previous accumulated value - with "pow(2)", and "+" as the two functions, you get L2 norm, with "abs()" and "+", you get L1 norm, etc...)

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

JIRA jira@apache.org

[jira] Updated: (MATH-312) RealVector interface could use some iterators (dense and sparse) and generic map() and collect() methods.

Reply Threaded More More options
Print post
Permalink
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/MATH-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jake Mannix updated MATH-312:
-----------------------------

    Attachment: MATH-312.patch

New patch. This one has no reference to the "nonzero default values" for sparse vectors, which can be addressed in another JIRA ticket.  This patch now only deals with the following files:

* src/main/java/org/apache/commons/math/linear/RealVector.java only methods added are iterator(), sparseIterator(), map(UnivariateRealFunction), mapToSelf(UnivariateRealFunction)
* src/main/java/org/apache/commons/math/analysis/UnivariateRealFunctions.java - a collection of UnivariateRealFunction wrappers to java.lang.Math static methods.
* src/main/java/org/apache/commons/math/linear/AbstractRealVector.java - implements the above methods, plus a ton more in the general case, so new implementations don't need to.
* src/main/java/org/apache/commons/math/linear/ArrayRealVector.java - primary change is to extend AbstractRealVector
* src/main/java/org/apache/commons/math/linear/OpenMapRealVector.java - ditto
* src/test/java/org/apache/commons/math/linear/AbstractRealVectorTest.java - tests for the new AbstractRealVector's new methods
* src/test/java/org/apache/commons/math/linear/ArrayRealVectorTest.java - update the inner interface
* src/test/java/org/apache/commons/math/linear/SparseRealVectorTest.java - ditto


> RealVector interface could use some iterators (dense and sparse) and generic map() and collect() methods.
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: MATH-312
>                 URL: https://issues.apache.org/jira/browse/MATH-312
>             Project: Commons Math
>          Issue Type: New Feature
>    Affects Versions: 2.0
>         Environment: all
>            Reporter: Jake Mannix
>             Fix For: 2.1
>
>         Attachments: MATH-312.patch, MATH-312.patch
>
>
> As discussed on the [math] list, there are other projects out there which would love to get a chance to standardize on using commons-math for things like linear algebra primitives, as it would build a common base to build upon.  But to do that, some well-known and used techniques for dealing with vectors, for one thing, are missing.  Most glaringly is the treatment of sparse vectors: giving no Iterator for non-default values means external clients lose the advantage of sparseness - only internal methods can skip around.  
> Extending the RealVector interface with sparse (and dense) iterator methods would fix this:
> {code}
>   double getDefaultValue();
>   Iterator<RealVector.Entry> iterator();
>   Iterator<RealVector.Entry> nonDefaultIterator();
> {code}
> but there is another way to deal with vector data as well: instead of passing iterators around, and worrying about all the lovely ConcurrentModification and unsupported "remove" methods (which aren't the end of the world), we can instead expose generic map functions:
> {code}
>   RealVector map(UnivariateRealFunction f);
>   RealVector mapToSelf(UnivariateRealFunction f);
> {code}
> where RealVector mapToSelf(UnivariateRealFunction), which applies the function to the vector's entries (checking whether the function preserves the default value up front allows it to chose between the sparse or dense iterator), and map just applies mapToSelf to a copy.
> This doesn't exhaust all possible places where Iterators could be used helpfully (there's also combining two vectors together via a {code}map(BinaryRealFunction, RealVector other){code} which could be specialized nonlinear forms of addition or subtraction, and {code}double collect(UnivariateRealFunction, BinaryRealFunction){code} which uses the iterates over all of the entries, applying the first unary function to each entry, and then applying the binary function to combine this value with the previous accumulated value - with "pow(2)", and "+" as the two functions, you get L2 norm, with "abs()" and "+", you get L1 norm, etc...)

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

JIRA jira@apache.org

[jira] Commented: (MATH-312) RealVector interface could use some iterators (dense and sparse) and generic map() and collect() methods.

Reply Threaded More More options
Print post
Permalink
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/MATH-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774695#action_12774695 ]

Bill Barker commented on MATH-312:
----------------------------------

The patch could use a lot more JavaDoc comments, but since they are mostly InheritDoc, I can live with putting them in myself.

What I do have a problem with is generalizing OpenMapRealVector to allow for different default values then zero.  This means that you get different results (in the sense of equals) if you try and operate with two vectors that have different default values depending on which is LHS and which is RHS.  I would need a use case to commit that can of worms.



> RealVector interface could use some iterators (dense and sparse) and generic map() and collect() methods.
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: MATH-312
>                 URL: https://issues.apache.org/jira/browse/MATH-312
>             Project: Commons Math
>          Issue Type: New Feature
>    Affects Versions: 2.0
>         Environment: all
>            Reporter: Jake Mannix
>             Fix For: 2.1
>
>         Attachments: MATH-312.patch, MATH-312.patch
>
>
> As discussed on the [math] list, there are other projects out there which would love to get a chance to standardize on using commons-math for things like linear algebra primitives, as it would build a common base to build upon.  But to do that, some well-known and used techniques for dealing with vectors, for one thing, are missing.  Most glaringly is the treatment of sparse vectors: giving no Iterator for non-default values means external clients lose the advantage of sparseness - only internal methods can skip around.  
> Extending the RealVector interface with sparse (and dense) iterator methods would fix this:
> {code}
>   double getDefaultValue();
>   Iterator<RealVector.Entry> iterator();
>   Iterator<RealVector.Entry> nonDefaultIterator();
> {code}
> but there is another way to deal with vector data as well: instead of passing iterators around, and worrying about all the lovely ConcurrentModification and unsupported "remove" methods (which aren't the end of the world), we can instead expose generic map functions:
> {code}
>   RealVector map(UnivariateRealFunction f);
>   RealVector mapToSelf(UnivariateRealFunction f);
> {code}
> where RealVector mapToSelf(UnivariateRealFunction), which applies the function to the vector's entries (checking whether the function preserves the default value up front allows it to chose between the sparse or dense iterator), and map just applies mapToSelf to a copy.
> This doesn't exhaust all possible places where Iterators could be used helpfully (there's also combining two vectors together via a {code}map(BinaryRealFunction, RealVector other){code} which could be specialized nonlinear forms of addition or subtraction, and {code}double collect(UnivariateRealFunction, BinaryRealFunction){code} which uses the iterates over all of the entries, applying the first unary function to each entry, and then applying the binary function to combine this value with the previous accumulated value - with "pow(2)", and "+" as the two functions, you get L2 norm, with "abs()" and "+", you get L1 norm, etc...)

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

JIRA jira@apache.org

[jira] Commented: (MATH-312) RealVector interface could use some iterators (dense and sparse) and generic map() and collect() methods.

Reply Threaded More More options
Print post
Permalink
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/MATH-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774710#action_12774710 ]

Jake Mannix commented on MATH-312:
----------------------------------

Bill, I thought I removed all reference to the nonzero default values idea, are you using the latest patch on there?

> RealVector interface could use some iterators (dense and sparse) and generic map() and collect() methods.
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: MATH-312
>                 URL: https://issues.apache.org/jira/browse/MATH-312
>             Project: Commons Math
>          Issue Type: New Feature
>    Affects Versions: 2.0
>         Environment: all
>            Reporter: Jake Mannix
>             Fix For: 2.1
>
>         Attachments: MATH-312.patch, MATH-312.patch
>
>
> As discussed on the [math] list, there are other projects out there which would love to get a chance to standardize on using commons-math for things like linear algebra primitives, as it would build a common base to build upon.  But to do that, some well-known and used techniques for dealing with vectors, for one thing, are missing.  Most glaringly is the treatment of sparse vectors: giving no Iterator for non-default values means external clients lose the advantage of sparseness - only internal methods can skip around.  
> Extending the RealVector interface with sparse (and dense) iterator methods would fix this:
> {code}
>   double getDefaultValue();
>   Iterator<RealVector.Entry> iterator();
>   Iterator<RealVector.Entry> nonDefaultIterator();
> {code}
> but there is another way to deal with vector data as well: instead of passing iterators around, and worrying about all the lovely ConcurrentModification and unsupported "remove" methods (which aren't the end of the world), we can instead expose generic map functions:
> {code}
>   RealVector map(UnivariateRealFunction f);
>   RealVector mapToSelf(UnivariateRealFunction f);
> {code}
> where RealVector mapToSelf(UnivariateRealFunction), which applies the function to the vector's entries (checking whether the function preserves the default value up front allows it to chose between the sparse or dense iterator), and map just applies mapToSelf to a copy.
> This doesn't exhaust all possible places where Iterators could be used helpfully (there's also combining two vectors together via a {code}map(BinaryRealFunction, RealVector other){code} which could be specialized nonlinear forms of addition or subtraction, and {code}double collect(UnivariateRealFunction, BinaryRealFunction){code} which uses the iterates over all of the entries, applying the first unary function to each entry, and then applying the binary function to combine this value with the previous accumulated value - with "pow(2)", and "+" as the two functions, you get L2 norm, with "abs()" and "+", you get L1 norm, etc...)

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.