• Add "-Dcom.sun.management.jmxremote" to your JVM.

For Application Server 10.1.3.x goto $ORACLE_HOME/opmn/config.

Edit opmn.xml. Add this to the process-type/start-paramters/java-options of the "home" or "oc4j-soa" or ...
Restart Application Server

For OC4J standalone edit $ORACLE_HOME/bin/oc4j.cmd or oc4j.sh, add

set JVMARGS=%OC4J_JVM_ARGS% -Dcom.sun.management.jmxremote=true
or
set OC4J_JVM_ARGS=-Dcom.sun.management.jmxremote=true before starting OC4J

For Application Server 11g (Technology PreviewTP3, TP4 or TP5)

- Go to your $JDEV_USER_DIR/system11.1.1.0.22.48.95/o.j2ee/embedded-oc4j/bin
( for TP5 go to $JDEV_USER_DIR/system11.1.1.0.30.50.54/o.j2ee/embedded-oc4j/bin)
- Edit the startsoa(.bat): add "-Dcom.sun.management.jmxremote" to the JAVA_VM paramter
- Start the server with this script directly or from within the JDeveloper 11g.

• Start JConsole and select the local process with "oc4j..."

JConsole screens:

• Start JVisualVM/VisualVM and select on the left side the process with "oc4j.jar"

JVisualVM/VisualVM screen:

• Tested also with JDK 1.6.0.10, see screen

• If you want to connect to a remote OC4J you can do this by starting JConsole with the following parameter:

jconsole
-J-Dcom.sun.management.jmxremote.ssl=false
-J-Dcom.sun.management.jmxremote.authenticate=true
-J-Djmx.remote.protocol.provider.pkgs=oracle.oc4j.admin.jmx.remote
-J-Djava.class.path=;%JAVA_HOME%\lib\jconsole.jar;%JAVA_HOME%lib\tools.jar;%ORACLE_HOME%\j2ee\home\lib\adminclient.jar;%ORACLE_HOME%\j2ee\home\oc4jclient.jar

Using the advanced connection tab, you can specify the direct JMX Service URL to identify the target server.
To connect remotely to an OC4J process the URL is of the format

OC4J standalone:

service:jmx:rmi://[oc4j-host]:[oc4j-port]
e.g. service:jmx:rmi://localhost:23791, User Name: oc4jadmin, password: welcome1

Oracle Application Server(OPMN managed):

service:jmx:rmi:///opmn://<hostname>:<opmn-port>/<container-name>
e.g. service:jmx:rmi:///opmn://localhost:6005/home, User Name: oc4jadmin, password: welcome1

• If you want to connect to a remote OC4J you can do this by starting JVisualVM or VisualVM with the following parameter:

jvisualvm
-J-Dcom.sun.management.jmxremote.ssl=false
-J-Dcom.sun.management.jmxremote.authenticate=true
-J-Djmx.remote.protocol.provider.pkgs=oracle.oc4j.admin.jmx.remote
--cp:a %ORACLE_HOME%\j2ee\home\lib\adminclient.jar;%ORACLE_HOME%\j2ee\home\oc4jclient.jar

Using File -> Add JMX Connection, and "Use security credential" (e.g. oc4jadmin/welcome1)

With the option "--cp:a" you define the classpath for the netbean. See "visualvm --help".
With the option "-J-Dnetbeans.logger.console=true" you can get the logging to the console.

Tested with JVisualVM: Java SE 6 Update 13 and 14 and VisualVM 1.1.1.